Skip to content

Commit

Permalink
feat: bump dependencies and readme (#8)
Browse files Browse the repository at this point in the history
* feat: bump dependencies and readme

* add citation cff

* small tweaks

* small tweaks to readme
  • Loading branch information
homanp authored Nov 25, 2024
1 parent 003bbac commit af9dad1
Show file tree
Hide file tree
Showing 4 changed files with 50 additions and 8 deletions.
8 changes: 8 additions & 0 deletions CITATION.cff
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
- family-names: "Ismail"
given-names: "Pelaseyd"
title: "Superagent PokerEval"
date-released: 2024-11-25
url: "https://github.com/superagent-ai/poker-eval"
40 changes: 37 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

A comprehensive tool for assessing AI agents performance in simulated poker environments. Written in Typescript.

[Getting Started](#getting-started) | [Examples](#examples) | [FAQ](#faq) | [Development](#development) | [Contributing](#contributing)
[Getting Started](#getting-started) | [Why Poker?](#why-poker) | [Leaderboard](#leaderboard) | [Examples](#examples)

</div>

Expand All @@ -24,24 +24,27 @@ npm i @superagent-ai/poker-eval
// index.ts

import { PokerGame } from "@superagent-ai/poker-eval";
import { Player, PlayerAction } from "@superagent-ai/poker-eval/dist/types"

// See example agent: https://github.com/superagent-ai/poker-eval/blob/main/examples/ai-sdk/agent.ts
import { generateAction } from "./agent";
import { Player, PlayerAction } from "../../src/types";


async function executeGameSimulation(numHands: number): Promise<void> {
// Setup AI players
const players: Player[] = [
{
name: "GPT 1",
action: async (state): Promise<PlayerAction> => {
// Use any model, framework or code to generate a response
const action = await generateAction(state);
return action;
},
},
{
name: "GPT 2",
action: async (state): Promise<PlayerAction> => {
// Use any model, framework or code to generate a response
const action = await generateAction(state);
return action;
},
Expand Down Expand Up @@ -82,10 +85,41 @@ In this example, the dataset shows the position of the player, their hole cards,

BB/100, or Big Blinds per 100 hands, is a common metric used in poker to measure a player's win rate. It represents the average number of big blinds a player wins or loses over 100 hands. To calculate BB/100, use the formula:

BB/100 = (Total bb_profit / Number of hands) * 100
`BB/100 = (Total bb_profit / Number of hands) * 100`

This formula provides a standardized measure of performance, allowing for comparison across different sessions or players by normalizing the win rate to a per-100-hands basis.


## Why Poker?
Poker combines elements of strategy, psychology, risk assessment, and partial information - perfect for testing an Agent's decision-making skills in complex, uncertain environments. Poker provides measurable KPIs like EV, BB/100, All-In adj BB/100 and VPIP. These KPIs are widely recognized standards, not created by a single company, making them ideal for objectively evaluating an Agent's decision-making skills.

We've specificalyy chosen No Limit Texas Holdem cash games and are officially calling the eval `NLTH`.

## Leaderboard
Coming soon...

## Examples
We've created some examples using populat agent frameworks you can use as inspiration (feel free to contribute):

- [Vercel AI SDK](https://github.com/superagent-ai/poker-eval/tree/main/examples/ai-sdk)
- [OpeaAI]() Coming soon
- [Mastra]() Coming soon
- [LlamaIndex]() Coming soon
- [Langchain]() Coming soon

## Citations
```json
{
"cff-version": "1.2.0",
"message": "If you use this software, please cite it as below.",
"authors": [
{
"family-names": "Ismail",
"given-names": "Pelaseyd"
}
],
"title": "Superagent PokerEval",
"date-released": "2024-11-25",
"url": "https://github.com/superagent-ai/poker-eval"
}
```
8 changes: 4 additions & 4 deletions examples/ai-sdk/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion examples/ai-sdk/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
"license": "ISC",
"dependencies": {
"@ai-sdk/openai": "^1.0.4",
"@superagent-ai/poker-eval": "^1.0.0",
"@superagent-ai/poker-eval": "^1.0.1",
"ai": "^4.0.3",
"dotenv": "^16.4.5",
"zod": "^3.23.8"
Expand Down

0 comments on commit af9dad1

Please sign in to comment.