Imagine an assistant that helps you figure out your finances, retrieves useful info, and does special functions—all in one go!
Using the Assistants API is simple. Just tell your assistant what to do, like having a conversation. It can understand different file formats, execute code (super techy!), and do all sorts of cool tricks. As Azure OpenAI adds more tools, your assistant can do even cooler things. It's a fun and easy way to add some brainpower to your GenAI app!
By leveraging the Assitants API, developers can create assistants with easy-to-follow instructions that use special tools to get things done. Right now, it has three superpowers: Code Interpreter (for doing techy stuff), Retrieval (for finding info), and Function calling (for making things happen). You can even mix these powers to create a super-assistant that can handle all sorts of tasks.
In this accelerator, our goal is to demonstrate the simplicity of integrating the Assistants API. We aim to empower you with the knowledge to grasp fundamental concepts and explore the full range of capabilities that the API offers.
At a high level, a typical integration of the Assistants API has the following flow:
- Create an Assistant in the API by defining its custom instructions and picking a model. If helpful, enable tools like Code Interpreter, Retrieval, and Function calling.
- Create a Thread when a user starts a conversation.
- Add Messages to the Thread as the user ask questions.
- Run the Assistant on the Thread to trigger responses. This automatically calls the relevant tools.
| Feature | Description |
|---|---|
| Threads | Contains Messages and automatically handles the truncation of content to fit within the context of a model. |
| Files | Allows importing file content of different file formats. Can be used in tools such as Retrieval scenarios and analysis with Code Interpreter. |
| Tools | Includes Code Interpreter, Knowledge Retrieval, and Function calling. - Code Interpreter allows executing code snippets. - Knowledge Retrieval automatically chunks and embeds content in files for augmented retrieval scenarios. - Function calling enables calling functions. |
| Tool composition | Enables using multiple tools in one Assistant. |
| Term | Definition |
|---|---|
| Assistant | An AI system specifically designed to utilize OpenAI's models and call tools. |
| Thread | A session of conversation between a user and an Assistant. Threads store Messages and automatically handle truncation to ensure that content fits within the context of the model. |
| Message | A piece of communication generated by either an Assistant or a user. Messages can contain text, images, and other files. Messages are stored as a list within a Thread. |
| Run | An instance of an Assistant being invoked on a Thread. The Assistant utilizes its configuration and the Messages within the Thread to perform tasks by calling models and tools. During a Run, the Assistant appends Messages to the Thread. |
| Run Step | A detailed record of the individual actions taken by the Assistant during a Run. These steps can include calling tools or generating Messages. Examining Run Steps provides insight into how the Assistant arrives at its final results. |
| Status | Description |
|---|---|
| queued | When Runs are initially created or when the required action is completed, they are placed in a queued status. They should quickly transition to an in_progress status. |
| completed | The Run has successfully finished! You can now access all the Messages added by the Assistant to the Thread, as well as the steps taken by the Run. You can also continue the conversation by adding more user Messages to the Thread and initiating another Run. |
| requires_action | When using the Function calling tool, the Run will transition to a required_action state once the model determines the names and arguments of the functions to be called. You must then execute those functions and submit the outputs before the Run can proceed. If the outputs are not provided before the expires_at timestamp (approximately 10 minutes after creation), the Run will move to an expired status. |
| expired | This occurs when the function calling outputs were not submitted before the expires_at timestamp and the Run expires. Additionally, if the Run takes too long to execute and exceeds the time specified in expires_at, our systems will mark the Run as expired. |
| cancelling | You can attempt to cancel an in_progress Run by using the Cancel Run endpoint. Once the cancellation attempt is successful, the status of the Run will change to cancelled. However, cancellation is not guaranteed and may not always be possible. |
| cancelled | The Run was successfully cancelled. |
| failed | The reason for the failure can be viewed by examining the last_error object in the Run. The timestamp for the failure will be recorded under failed_at. |
Reference:
Note: the following code showcases the foundational Assistant concepts such as Assistants, Threads, Messages, Runs and the Assistant lifecycle.
import time
from dotenv import load_dotenv
from openai import AzureOpenAI
# Load the environment variables - These are secrets.
load_dotenv()
api_URI = os.getenv("OPENAI_URI")
api_KEY = os.getenv("OPENAI_KEY")
api_version = os.getenv("OPENAI_VERSION")
deployment_name = os.getenv("OPENAI_DEPLOYMENT_NAME")
# Create an OpenAI Azure client
client = AzureOpenAI(api_key=api_key,
api_version=api_version,
azure_endpoint=api_endpoint)
# Create an Asssitant with the code_interpreter tool
assistant = client.beta.assistants.create(
name="Math Tutor",
instructions="You are a personal math tutor. Write and run code to answer math questions.",
tools=[{"type": "code_interpreter"}],
model=deployment_name,
)
# Create a Thread
thread = client.beta.threads.create()
# Create a Message
message = client.beta.threads.messages.create(
thread_id=thread.id,
role="user",
content="I need to solve the equation `3x + 11 = 14`. Can you help me?",
)
# Create a Run
run = client.beta.threads.runs.create(
thread_id=thread.id,
assistant_id=assistant.id,
instructions="Please address the user as Jane Doe. The user has a premium account.",
)
# Check the status of a Run
print("checking assistant status. ")
while True:
# Retrieve a Run by Thread and Run id
run = client.beta.threads.runs.retrieve(thread_id=thread.id, run_id=run.id)
if run.status == "completed":
# Get the messages for a thread
messages = client.beta.threads.messages.list(thread_id=thread.id)
# Print the messages
print("messages: ")
for message in messages:
assert message.content[0].type == "text"
print({"role": message.role, "message": message.content[0].text.value})
# Dispose of the assistant
client.beta.assistants.delete(assistant.id)
# Dispose of the thread
client.beta.threads.delete(thread.id)
break
elif run.status = "requires_action":
# handle funcation calling and continue with the execution
pass
elif run.status = "expired" or run.status=="failed" or run.status=="cancelled":
# run failed, expired, or was cancelled
break
else:
print("in progress...")
time.sleep(5)Note: Some Assistants in the notebooks folder have the ability to send emails and create appointments in Outlook. This functionality is achieved by using Function calling, which in turn triggers an HTTP Azure Logic App. The process of creating the Logic App is not covered in this guide. However, if you are interested in learning more about creating Azure Logic Apps, you can follow this link.
| Topic | Description |
|---|---|
| Math Tutor | Showcases the foundational concepts of Assistants such as Threads, Messages, Runs, Tools, and lifecycle management. |
| Financial Assistant | Function Calling with Yfinance to get latest stock prices. Summarization of user provided article. Extract country info from article, extract country, capital and other aspects, and call an API to get more information about each country. |
| Failed Banks | Using Assistant tools Code Interpreter and Function calling, this Assistant can get a CSV file, gather a list of failed banks by state, and generate a chart to visually represent the data. |
| Wind Farm | Utilizing Assistant tools such as the Code Interpreter and Function calling, this bot is capable of retrieving a CSV file that illustrates turbine wind speed, voltage, and the last maintenance date. It assists you in reviewing the file contents and aids in determining whether a specific turbine is in need of maintenance. |
| Sales Assistant | Showcases how you can create an Assistant adept at managing various tasks, such as handling relational data across different files and supporting multiple users simultaneously within the same Assistant across distinct threads. |
| Autogen Assistants | Get started with Assistants on Autogen, with examples of basic assistants, code interpreter, retrieval and function calling, entirely managed by the Autogen framework. |
| File Search Assistant | Leverage File Search with Assistants, with examples of retrieval over a document containing benefits information. |
- An Azure subscription
- Create and deploy an Azure OpenAI Service Resource
- In Azure create OpenAI endpoints for:
- Model version: GPT 4 (1106-Preview)
- API version: 2024-02-15-preview
- Copy the deployment name, endpoint, and API key information
- In Azure create OpenAI endpoints for:
- Clone this repo
- Make sure the VS Code is installed as well as the VS Code Polyglot extension
- Python 3.7+
- Create/rename
\.envfile and insert your Azure Open AI credentials:
OPENAI_URI=https://<NAME>.openai.azure.com/
OPENAI_KEY=<API_KEY>
OPENAI_VERSION=2024-02-15-preview
OPENAI_GPT_DEPLOYMENT=<GPT_DEPLOYMENT_NAME>
OPENAI_ADA_DEPLOYMENT=<ADA_DEPLOYMENT_NAME>
- Open a Notebook and click play on the cells