Skip to content

Commit

Permalink
More AI server docs.
Browse files Browse the repository at this point in the history
  • Loading branch information
Layoric committed Oct 2, 2024
1 parent ddea290 commit 8b9cded
Show file tree
Hide file tree
Showing 4 changed files with 156 additions and 7 deletions.
Empty file.
35 changes: 28 additions & 7 deletions MyApp/_pages/ai-server/usage/chat.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,10 @@ var response = client.Post(new CreateOpenAiChat {
},
MaxTokens = 50
},
RefId = "<unique-id>",
Provider = "openai",
ReplyTo = "https://myapp.com/reply",
Sync = true
RefId = "<unique-id>", // Optional
Provider = "openai", // Optional
ReplyTo = "https://myapp.com/reply", // Optional
Sync = true // Optional
});
```

Expand All @@ -33,7 +33,7 @@ Additional optional features on the request to enhance the usage of AI Server in
- **Provider**: Force the request to use a specific provider, overriding the selection logic.
- **ReplyTo**: A HTTP URL to send the response to on completion of the request.
- **Tag**: A tag to help categorize the request for easier tracking.
- **Sync**: A boolean value to determine if the request should be processed synchronously (default is asynchronous returning job info).
- **Sync**: A boolean value to determine if the request should be processed synchronously (default is `false`).

These additional request properties are consistent across all AI Server endpoints, providing a common interface for all AI services.

Expand All @@ -42,7 +42,28 @@ These additional request properties are consistent across all AI Server endpoint
One advantage of using AI Server is that it provides a common set of request DTOs in 11 different languages that are compatible with OpenAI's API. This allows you to switch between OpenAI and AI Server without changing your client code.
This means you can switch to using typed APIs in your preferred language with your existing service providers OpenAI compatible APIs, and optionally switch to AI Server when you're ready to self-host your AI services for better value.

::include ai-server/cs/open-ai-requests-1.cs.md::
```csharp
// Using OpenAI API directly via JsonApiClient
var client = new JsonApiClient("https://api.openai.com/v1");
client.AddHeader("Authorization", "Bearer " + Environment.GetEnvironmentVariable("OPENAI_API_KEY"));

// Using AI Server DTOs with OpenAI API
var request = new OpenAiChat {
Model = "gpt-4-turbo",
Messages = new List<OpenAiMessage> {
new OpenAiMessage { Role = "system", Content = "You are a helpful AI assistant." },
new OpenAiMessage { Role = "user", Content = "How do LLMs work?" }
},
MaxTokens = 50
};

// Typed response DTO
var response = await client.PostAsync<OpenAiChatResponse>("/chat/completions", request);
```

This shows usage of the `OpenAiChat` request DTO directly with OpenAI's API using the ServiceStack `JsonApiClient`, so you get the benefits of using typed APIs in your preferred language with your existing service providers OpenAI compatible APIs.




This shows usage of the `OpenAiChat` request DTO directly with OpenAI's API. The same request DTO can be used with AI Server's `/api/CreateOpenAiChat` endpoint, populated the same way.

70 changes: 70 additions & 0 deletions MyApp/_pages/ai-server/usage/image-endpoints.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
---
title: "Image to Image"
description: "Generating images from images with AI Server"
---

# Image to Image

AI Server has built-in ComfyUI workflows for doing image-to-image generation tasks like inpainting. This takes an image as input and generates a new image based on the input image and any additional prompts you provide.

## Other Image to Image Tasks

In addition to just Image to Image, AI Server also supports:

- **ImageWithMask**: This task takes an image and a mask as input and uses the provided prompts only for the matching masked area.
- **ImageUpscale**: This task takes an image and upscales it to a higher resolution using the provided prompts (2x only currently).
- **ImageToText**: This task takes an image and generates text based on the image content.

## Using Image to Image

To generate an image from an image, you can use the `ImageToImage` request:

```csharp
var request = new ImageToImage()
{
PositivePrompt = "A beautiful sunset over the ocean",
NegativePrompt = "A pixelated, low-quality image",
Sync = true
};

var response = client.PostFilesWithRequest<GenerationResponse>(
request,
[new UploadFile("image", File.OpenRead("sunset.jpg"), "sunset.jpg")]
);
response.Outputs[0].Url.DownloadFileTo("ocean-sunset.webp");
```

A similar pattern can be used for the `ImageWithMask` request:

```csharp
var request = new ImageWithMask()
{
PositivePrompt = "A beautiful sunset over the ocean",
NegativePrompt = "A pixelated, low-quality image",
Sync = true
};

var response = client.PostFilesWithRequest<GenerationResponse>(
request,
[new UploadFile("image", File.OpenRead("sunset.jpg"), "sunset.jpg"),
new UploadFile("mask", File.OpenRead("mask.jpg"), "mask.jpg")]
);
response.Outputs[0].Url.DownloadFileTo("ocean-sunset.webp");
```

The `ImageUpscale` request is similar, but only requires the image file:

```csharp
var request = new ImageUpscale()
{
Sync = true
};

var response = client.PostFilesWithRequest<GenerationResponse>(
request,
[new UploadFile("image", File.OpenRead("low-res.jpg"), "low-res.jpg")]
);
response.Outputs[0].Url.DownloadFileTo("high-res.webp");
```

Upscaling is currently limited to 2x the original resolution, and it requires ComfyUI has downloaded the necessary model "RealESRGAN_x2.pth" to perform the upscale.
58 changes: 58 additions & 0 deletions MyApp/_pages/ai-server/usage/text-to-image.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
---
title: "Text to Image"
description: "Generating images from text with AI Server"
---

# Text to Image

As well as interacting with LLMs to generate text, AI Server can also generate images from text from a few different providers.

- **DALL-E**
- **Replicate API**
- **Comfy UI**

## DALL-E

When you configure AI Server with an OpenAI API key, you can use the DALL-E model to generate images from text.

You can control these providers via the Admin Dashboard in AI Server if you want to enable or disable them on a per model basis.

## Replicate API

Replicate is another provider that can generate images from text. You can configure Replicate in the Admin Dashboard in AI Server or include the `REPLICATE_API_KEY` in your `.env` file during the first run of AI Server.

Replicate provides access to the Flux family of models, which can generate images from text using:

- **Flux Schnell**
- **Flux Dev**
- **Flux Pro**

## Comfy UI

Comfy UI is a self-hosted agent that can process image requests and other modalities. You can configure Comfy UI in the Admin Dashboard in AI Server after you have set a ComfyUI instance using [the related ComfyUI Extension](https://github.com/ServiceStack/agent-comfy).

When configuring the Comfy AI Provider, you can provide the URL of your ComfyUI instance, and any API key required to authenticate with it, and you will get a list of the models available in your ComfyUI instance to enable for the provider.

## Using Text to Image

Once you have configured your AI Server with the providers you want to use, you can make requests to the AI Server API to generate images from text.

```csharp
var request = new TextToImage()
{
Height = 768,
Width = 768,
Model = "flux-schnell",
PositivePrompt = "A happy llama",
NegativePrompt = "bad quality, blurry image",
Sync = true
};

var response = await client.PostAsync(request);
response.Outputs[0].Url.DownloadFileTo("llama.webp");
```

This request will generate an image of a happy llama using the Flux Schnell model. The `PositivePrompt` and `NegativePrompt` properties are used to guide the model on what to generate, and what to avoid. The `Sync` property is used to determine if the request should be processed synchronously or asynchronously. By default, requests are processed asynchronously.

Flux Schnell is also available in the Comfy UI agent, so you can use the same model with multiple providers, or switch between providers without changing your client code.

0 comments on commit 8b9cded

Please sign in to comment.