-
Notifications
You must be signed in to change notification settings - Fork 120
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Use CI on v2 branch * branch * add docs, move scoring functions to scoring.ts, move experiment naming to utils.ts * add initStagehand.ts * break up index.evals.ts and utils into smaller files * export LogLineEval * typing * follow StagehandConfig pattern * choose api key based on model name * stagehand.act -> page.act (#326) * need to actually move to act to page now * move act -> page * fix e2e * fix tests * readme * changeset * package json and changeset * don't fail on combo evals * Add act evals on `stagehand.page` (#328) * move act evals to stagehand.page * add basic act and make act necessary in type * move extract and observe to page (#329) * move act evals to stagehand.page * add basic act and make act necessary in type * move extract and observe * example * changeset * More playwright tests (#330) * add docs, move scoring functions to scoring.ts, move experiment naming to utils.ts * add initStagehand.ts * break up index.evals.ts and utils into smaller files * export LogLineEval * typing * follow StagehandConfig pattern * choose api key based on model name * Use CI on v2 branch * branch * stagehand.page tests * dont run on BB * prettier * pls dont fail * headless --------- Co-authored-by: Anirudh Kamath <[email protected]> * add extract evals for stagehand.page (#331) * add extract evals for stagehand.page * fix typign * smh i didn't actually run extract * add observe page evals (#332) * change stagehand.observe to stagehand.page.observe in evals * changeset * Browsercontext playwright tests (#334) * add docs, move scoring functions to scoring.ts, move experiment naming to utils.ts * add initStagehand.ts * break up index.evals.ts and utils into smaller files * export LogLineEval * typing * follow StagehandConfig pattern * choose api key based on model name * Use CI on v2 branch * branch * BrowserContext tests * file path --------- Co-authored-by: Anirudh Kamath <[email protected]> * changeset minor * ci yml --------- Co-authored-by: seanmcguire12 <[email protected]> Co-authored-by: Sean McGuire <[email protected]>
- Loading branch information
3 people
authored
Dec 24, 2024
1 parent
cc46f34
commit cd23fa3
Showing
93 changed files
with
2,540 additions
and
820 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@browserbasehq/stagehand": minor | ||
| --- | ||
|
|
||
| Move stagehand.act() -> stagehand.page.act() and deprecate stagehand.act() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,5 @@ | ||
| --- | ||
| "@browserbasehq/stagehand": patch | ||
| "@browserbasehq/stagehand": minor | ||
| --- | ||
|
|
||
| We now wrap playwright page/context within StagehandPage and StagehandContext objects. This helps us augment the Stagehand experience by being able to augment the underlying Playwright |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@browserbasehq/stagehand": minor | ||
| --- | ||
|
|
||
| moves extract and act -> page and deprecates stagehand.extract and stagehand.observe |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -110,9 +110,10 @@ const stagehand = new Stagehand({ | |
|
|
||
| ```javascript | ||
| await stagehand.init(); | ||
| await stagehand.page.goto("https://github.com/browserbase/stagehand"); | ||
| await stagehand.act({ action: "click on the contributors" }); | ||
| const contributor = await stagehand.extract({ | ||
| const page = stagehand.page; | ||
| await page.goto("https://github.com/browserbase/stagehand"); | ||
| await page.act({ action: "click on the contributors" }); | ||
| const contributor = await page.extract({ | ||
| instruction: "extract the top contributor", | ||
| schema: z.object({ | ||
| username: z.string(), | ||
|
|
@@ -209,6 +210,9 @@ This constructor is used to create an instance of Stagehand. | |
|
|
||
| `act()` allows Stagehand to interact with a web page. Provide an `action` like `"search for 'x'"`, or `"select the cheapest flight presented"` (small atomic goals perform the best). | ||
|
|
||
| > [!WARNING] | ||
| > `act()` on the Stagehand instance is deprecated and will be removed in the next major version. Use `stagehand.page.act()` instead. | ||
| - **Arguments:** | ||
|
|
||
| - `action`: a `string` describing the action to perform | ||
|
|
@@ -229,18 +233,18 @@ This constructor is used to create an instance of Stagehand. | |
|
|
||
| ```javascript | ||
| // Basic usage | ||
| await stagehand.act({ action: "click on add to cart" }); | ||
| await stagehand.page.act({ action: "click on add to cart" }); | ||
|
|
||
| // Using variables | ||
| await stagehand.act({ | ||
| await stagehand.page.act({ | ||
| action: "enter %username% into the username field", | ||
| variables: { | ||
| username: "[email protected]", | ||
| }, | ||
| }); | ||
|
|
||
| // Multiple variables | ||
| await stagehand.act({ | ||
| await stagehand.page.act({ | ||
| action: "fill in the form with %username% and %password%", | ||
| variables: { | ||
| username: "john.doe", | ||
|
|
@@ -253,6 +257,9 @@ This constructor is used to create an instance of Stagehand. | |
|
|
||
| `extract()` grabs structured text from the current page using [zod](https://github.com/colinhacks/zod). Given instructions and `schema`, you will receive structured data. Unlike some extraction libraries, stagehand can extract any information on a page, not just the main article contents. | ||
|
|
||
| > [!WARNING] | ||
| > `extract()` on the Stagehand instance is deprecated and will be removed in the next major version. Use `stagehand.page.extract()` instead. | ||
| - **Arguments:** | ||
|
|
||
| - `instruction`: a `string` providing instructions for extraction | ||
|
|
@@ -268,7 +275,7 @@ This constructor is used to create an instance of Stagehand. | |
|
|
||
| - **Example:** | ||
| ```javascript | ||
| const price = await stagehand.extract({ | ||
| const price = await stagehand.page.extract({ | ||
| instruction: "extract the price of the item", | ||
| schema: z.object({ | ||
| price: z.number(), | ||
|
|
@@ -278,6 +285,9 @@ This constructor is used to create an instance of Stagehand. | |
|
|
||
| #### `observe()` | ||
|
|
||
| > [!WARNING] | ||
| > `observe()` on the Stagehand instance is deprecated and will be removed in the next major version. Use `stagehand.page.observe()` instead. | ||
| > [!NOTE] | ||
| > `observe()` currently only evaluates the first chunk in the page. | ||
|
|
@@ -301,7 +311,7 @@ If you are looking for a specific element, you can also pass in an instruction t | |
|
|
||
| - **Example:** | ||
| ```javascript | ||
| const actions = await stagehand.observe(); | ||
| const actions = await stagehand.page.observe(); | ||
| ``` | ||
|
|
||
| #### `close()` | ||
|
|
@@ -409,9 +419,9 @@ Prompting Stagehand is more literal and atomic than other higher level framework | |
| - **Use specific and concise actions** | ||
|
|
||
| ```javascript | ||
| await stagehand.act({ action: "click the login button" }); | ||
| await stagehand.page.act({ action: "click the login button" }); | ||
|
|
||
| const productInfo = await stagehand.extract({ | ||
| const productInfo = await stagehand.page.extract({ | ||
| instruction: "find the red shoes", | ||
| schema: z.object({ | ||
| productName: z.string(), | ||
|
|
@@ -426,22 +436,22 @@ Instead of combining actions: | |
|
|
||
| ```javascript | ||
| // Avoid this | ||
| await stagehand.act({ action: "log in and purchase the first item" }); | ||
| await stagehand.page.act({ action: "log in and purchase the first item" }); | ||
| ``` | ||
|
|
||
| Split them into individual steps: | ||
|
|
||
| ```javascript | ||
| await stagehand.act({ action: "click the login button" }); | ||
| await stagehand.page.act({ action: "click the login button" }); | ||
| // ...additional steps to log in... | ||
| await stagehand.act({ action: "click on the first item" }); | ||
| await stagehand.act({ action: "click the purchase button" }); | ||
| await stagehand.page.act({ action: "click on the first item" }); | ||
| await stagehand.page.act({ action: "click the purchase button" }); | ||
| ``` | ||
|
|
||
| - **Use `observe()` to get actionable suggestions from the current page** | ||
|
|
||
| ```javascript | ||
| const actions = await stagehand.observe(); | ||
| const actions = await stagehand.page.observe(); | ||
| console.log("Possible actions:", actions); | ||
| ``` | ||
|
|
||
|
|
@@ -451,21 +461,21 @@ console.log("Possible actions:", actions); | |
|
|
||
| ```javascript | ||
| // Too vague | ||
| await stagehand.act({ action: "find something interesting on the page" }); | ||
| await stagehand.page.act({ action: "find something interesting on the page" }); | ||
| ``` | ||
|
|
||
| - **Combine multiple actions into one instruction** | ||
|
|
||
| ```javascript | ||
| // Avoid combining actions | ||
| await stagehand.act({ action: "fill out the form and submit it" }); | ||
| await stagehand.page.act({ action: "fill out the form and submit it" }); | ||
| ``` | ||
|
|
||
| - **Expect Stagehand to perform high-level planning or reasoning** | ||
|
|
||
| ```javascript | ||
| // Outside Stagehand's scope | ||
| await stagehand.act({ action: "book the cheapest flight available" }); | ||
| await stagehand.page.act({ action: "book the cheapest flight available" }); | ||
| ``` | ||
|
|
||
| By following these guidelines, you'll increase the reliability and effectiveness of your web automations with Stagehand. Remember, Stagehand excels at executing precise, well-defined actions so keeping your instructions atomic will lead to the best outcomes. | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
| import process from "process"; | ||
| import { EvalCategorySchema } from "../types/evals"; | ||
|
|
||
| // Extract command-line arguments passed to this script. | ||
| const args = process.argv.slice(2); | ||
|
|
||
| /** | ||
| * The default categories of evaluations to run if none is specified. | ||
| * These categories represent different styles or types of tasks. | ||
| */ | ||
| const DEFAULT_EVAL_CATEGORIES = process.env.EVAL_CATEGORIES | ||
| ? process.env.EVAL_CATEGORIES.split(",") | ||
| : [ | ||
| "observe", | ||
| "act", | ||
| "combination", | ||
| "extract", | ||
| "experimental", | ||
| "text_extract", | ||
| ]; | ||
|
|
||
| /** | ||
| * Determine which extraction method to use for tasks that involve extraction. | ||
| * By default, "domExtract" is used. However, if a `--extract-method=<method>` | ||
| * argument is provided, it will override the default. | ||
| */ | ||
| let extractMethod = "domExtract"; | ||
| const extractMethodArg = args.find((arg) => | ||
| arg.startsWith("--extract-method="), | ||
| ); | ||
| if (extractMethodArg) { | ||
| extractMethod = extractMethodArg.split("=")[1]; | ||
| } | ||
|
|
||
| // Set the extraction method in the process environment so tasks can reference it. | ||
| process.env.EXTRACT_METHOD = extractMethod; | ||
| const useTextExtract = process.env.EXTRACT_METHOD === "textExtract"; | ||
|
|
||
| /** | ||
| * Variables for filtering which tasks to run: | ||
| * - `filterByCategory`: if provided, only tasks that belong to this category will be run. | ||
| * - `filterByEvalName`: if provided, only the task with this name will be run. | ||
| */ | ||
| let filterByCategory: string | null = null; | ||
| let filterByEvalName: string | null = null; | ||
|
|
||
| /** | ||
| * Check the first argument: | ||
| * - If it is "category", the next argument should be the category name. | ||
| * - Otherwise, assume it is a specific evaluation (task) name. | ||
| */ | ||
| if (args.length > 0) { | ||
| if (args[0].toLowerCase() === "category") { | ||
| filterByCategory = args[1]; | ||
| if (!filterByCategory) { | ||
| console.error("Error: Category name not specified."); | ||
| process.exit(1); | ||
| } | ||
| // Validate that the category is one of the known ones. | ||
| try { | ||
| EvalCategorySchema.parse(filterByCategory); | ||
| } catch { | ||
| console.error( | ||
| `Error: Invalid category "${filterByCategory}". Valid categories are: ${DEFAULT_EVAL_CATEGORIES.join(", ")}`, | ||
| ); | ||
| process.exit(1); | ||
| } | ||
| } else { | ||
| // Otherwise, treat it as a filter by evaluation name. | ||
| filterByEvalName = args[0]; | ||
| } | ||
| } | ||
|
|
||
| export { | ||
| filterByCategory, | ||
| filterByEvalName, | ||
| useTextExtract, | ||
| DEFAULT_EVAL_CATEGORIES, | ||
| }; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.