Skip to content

Commit

Permalink
docs(copilot): improve docs.
Browse files Browse the repository at this point in the history
  • Loading branch information
asafkorem committed Sep 29, 2024
1 parent d449ab4 commit fd3b7b5
Show file tree
Hide file tree
Showing 2 changed files with 14 additions and 82 deletions.
85 changes: 4 additions & 81 deletions docs/api/copilot.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ Detox Copilot is in active development, and APIs are subject to change in future

Detox Copilot exposes a simple API that integrates seamlessly with your Detox tests. It requires minimal setup and allows you to perform complex testing operations by simply describing them in natural language.

For a more detailed guide on integrating Detox Copilot in your tests, refer to the [Detox Copilot Guide].

## Methods

- [`copilot.init()`](#copilotinitprompthandler)
Expand Down Expand Up @@ -90,85 +92,6 @@ interface PromptHandler {
}
```

### Implementing a `PromptHandler`

You need to implement this interface to connect Detox Copilot with your LLM service. Below is an example using OpenAI's GPT-4 API.

**Example:**

```javascript
const { Configuration, OpenAIApi } = require('openai');

class OpenAIPromptHandler {
constructor(apiKey) {
const configuration = new Configuration({ apiKey });
this.openai = new OpenAIApi(configuration);
}

async runPrompt(prompt, image) {
const messages = [
{ role: 'system', content: 'You are a test automation assistant.' },
{ role: 'user', content: prompt },
];

// Handle image if supported
if (image && this.isSnapshotImageSupported()) {
// Implement image handling as per LLM requirements
}

const response = await this.openai.createChatCompletion({
model: 'gpt-4',
messages,
});

return response.data.choices[0].message.content;
}

isSnapshotImageSupported() {
return false; // Set to true if your LLM supports images
}
}

module.exports = OpenAIPromptHandler;
```

## Example Usage

Here is a complete example of how to use Detox Copilot in a test:

```javascript
const { device, copilot } = require('detox');
const OpenAIPromptHandler = require('./OpenAIPromptHandler');

describe('Login Flow', () => {
beforeAll(async () => {
await device.launchApp();
const promptHandler = new OpenAIPromptHandler('YOUR_OPENAI_API_KEY');
copilot.init(promptHandler);
});

it('should log in successfully', async () => {
await copilot.perform(
'Start the application',
'Tap on the "Login" button',
'Enter "[email protected]" into the email field',
'Enter "password123" into the password field',
'Press the "Submit" button',
'The welcome message "Hello, User!" should be displayed'
);
});
});
```

## Additional Notes

- **LLM Compatibility:** Detox Copilot is LLM-agnostic. While it can work with any LLM service, we recommend using advanced models like **Sonnet 3.5** or **GPT-4o** for better performance.

- **Visual Assertions:** Detox Copilot leverages the app's visual context (view hierarchy and snapshots) to enable the LLM to perform visual assertions and understand the UI state.

- **Core Library:** For more advanced configurations and to explore extending Detox Copilot to other testing frameworks, refer to the [detox-copilot core library](https://github.com/wix-incubator/detox-copilot).

- **Guide:** For detailed usage instructions and best practices, refer to the [Detox Copilot Guide]

You can refer to the [Detox Copilot Guide] for an example of implementing a `PromptHandler` for OpenAI's service.

[Detox Copilot Guide]: /docs/guides/testing-with-copilot
[Detox Copilot Guide]: /docs/guide/testing-with-copilot
11 changes: 10 additions & 1 deletion docs/guide/testing-with-copilot.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ It interprets these instructions and translates them into Detox commands. This g
Before you begin, ensure that your Detox environment is properly set up.
If you need assistance with the setup, refer to the [Detox Getting Started Guide](docs/introduction/getting-started/).

## Step 1: Implementing a `PromptHandler`
## Step 1: Implementing a PromptHandler

The `PromptHandler` is a crucial component that interfaces with your LLM service.
Below is an example of how to implement a `PromptHandler` using OpenAI's GPT-4 API.
Expand Down Expand Up @@ -203,3 +203,12 @@ To make the most out of Detox Copilot, consider the following best practices whe
**Q**: How can I provide feedback or contribute to Detox Copilot?
**A**: Contributions are welcome! Visit the [Detox Copilot GitHub Repository](https://github.com/wix-incubator/detox-copilot) to open issues or pull requests if they are relevant to the core-library functionality or open a it under [Detox repository](https://github.com/wix/Detox) if it is related to Detox-Copilot integration or if you are not sure where the issue should be opened.
---
**Q**: These are heavy operations for a test (uploading images, calling an LLM). Do you optimize it in any way?
**A**: Detox Copilot is designed to avoid unnecessary calls to the LLM service and optimize performance using static cache that is based on the current state of the app.
This minimizes the number of calls to the LLM service and reduces latency.
However, you can optimize your `PromptHandler` implementation to reduce latency and improve response times (e.g., by reducing the image size or implementing a server-side cache).
We have plans to optimize even further by introducing more advanced caching mechanisms for better performance.

0 comments on commit fd3b7b5

Please sign in to comment.