generated from NetCoreTemplates/razor-ssg
-
Notifications
You must be signed in to change notification settings - Fork 3
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
4 changed files
with
156 additions
and
7 deletions.
There are no files selected for viewing
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,70 @@ | ||
--- | ||
title: "Image to Image" | ||
description: "Generating images from images with AI Server" | ||
--- | ||
|
||
# Image to Image | ||
|
||
AI Server has built-in ComfyUI workflows for doing image-to-image generation tasks like inpainting. This takes an image as input and generates a new image based on the input image and any additional prompts you provide. | ||
|
||
## Other Image to Image Tasks | ||
|
||
In addition to just Image to Image, AI Server also supports: | ||
|
||
- **ImageWithMask**: This task takes an image and a mask as input and uses the provided prompts only for the matching masked area. | ||
- **ImageUpscale**: This task takes an image and upscales it to a higher resolution using the provided prompts (2x only currently). | ||
- **ImageToText**: This task takes an image and generates text based on the image content. | ||
|
||
## Using Image to Image | ||
|
||
To generate an image from an image, you can use the `ImageToImage` request: | ||
|
||
```csharp | ||
var request = new ImageToImage() | ||
{ | ||
PositivePrompt = "A beautiful sunset over the ocean", | ||
NegativePrompt = "A pixelated, low-quality image", | ||
Sync = true | ||
}; | ||
|
||
var response = client.PostFilesWithRequest<GenerationResponse>( | ||
request, | ||
[new UploadFile("image", File.OpenRead("sunset.jpg"), "sunset.jpg")] | ||
); | ||
response.Outputs[0].Url.DownloadFileTo("ocean-sunset.webp"); | ||
``` | ||
|
||
A similar pattern can be used for the `ImageWithMask` request: | ||
|
||
```csharp | ||
var request = new ImageWithMask() | ||
{ | ||
PositivePrompt = "A beautiful sunset over the ocean", | ||
NegativePrompt = "A pixelated, low-quality image", | ||
Sync = true | ||
}; | ||
|
||
var response = client.PostFilesWithRequest<GenerationResponse>( | ||
request, | ||
[new UploadFile("image", File.OpenRead("sunset.jpg"), "sunset.jpg"), | ||
new UploadFile("mask", File.OpenRead("mask.jpg"), "mask.jpg")] | ||
); | ||
response.Outputs[0].Url.DownloadFileTo("ocean-sunset.webp"); | ||
``` | ||
|
||
The `ImageUpscale` request is similar, but only requires the image file: | ||
|
||
```csharp | ||
var request = new ImageUpscale() | ||
{ | ||
Sync = true | ||
}; | ||
|
||
var response = client.PostFilesWithRequest<GenerationResponse>( | ||
request, | ||
[new UploadFile("image", File.OpenRead("low-res.jpg"), "low-res.jpg")] | ||
); | ||
response.Outputs[0].Url.DownloadFileTo("high-res.webp"); | ||
``` | ||
|
||
Upscaling is currently limited to 2x the original resolution, and it requires ComfyUI has downloaded the necessary model "RealESRGAN_x2.pth" to perform the upscale. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,58 @@ | ||
--- | ||
title: "Text to Image" | ||
description: "Generating images from text with AI Server" | ||
--- | ||
|
||
# Text to Image | ||
|
||
As well as interacting with LLMs to generate text, AI Server can also generate images from text from a few different providers. | ||
|
||
- **DALL-E** | ||
- **Replicate API** | ||
- **Comfy UI** | ||
|
||
## DALL-E | ||
|
||
When you configure AI Server with an OpenAI API key, you can use the DALL-E model to generate images from text. | ||
|
||
You can control these providers via the Admin Dashboard in AI Server if you want to enable or disable them on a per model basis. | ||
|
||
## Replicate API | ||
|
||
Replicate is another provider that can generate images from text. You can configure Replicate in the Admin Dashboard in AI Server or include the `REPLICATE_API_KEY` in your `.env` file during the first run of AI Server. | ||
|
||
Replicate provides access to the Flux family of models, which can generate images from text using: | ||
|
||
- **Flux Schnell** | ||
- **Flux Dev** | ||
- **Flux Pro** | ||
|
||
## Comfy UI | ||
|
||
Comfy UI is a self-hosted agent that can process image requests and other modalities. You can configure Comfy UI in the Admin Dashboard in AI Server after you have set a ComfyUI instance using [the related ComfyUI Extension](https://github.com/ServiceStack/agent-comfy). | ||
|
||
When configuring the Comfy AI Provider, you can provide the URL of your ComfyUI instance, and any API key required to authenticate with it, and you will get a list of the models available in your ComfyUI instance to enable for the provider. | ||
|
||
## Using Text to Image | ||
|
||
Once you have configured your AI Server with the providers you want to use, you can make requests to the AI Server API to generate images from text. | ||
|
||
```csharp | ||
var request = new TextToImage() | ||
{ | ||
Height = 768, | ||
Width = 768, | ||
Model = "flux-schnell", | ||
PositivePrompt = "A happy llama", | ||
NegativePrompt = "bad quality, blurry image", | ||
Sync = true | ||
}; | ||
|
||
var response = await client.PostAsync(request); | ||
response.Outputs[0].Url.DownloadFileTo("llama.webp"); | ||
``` | ||
|
||
This request will generate an image of a happy llama using the Flux Schnell model. The `PositivePrompt` and `NegativePrompt` properties are used to guide the model on what to generate, and what to avoid. The `Sync` property is used to determine if the request should be processed synchronously or asynchronously. By default, requests are processed asynchronously. | ||
|
||
Flux Schnell is also available in the Comfy UI agent, so you can use the same model with multiple providers, or switch between providers without changing your client code. | ||
|