Skip to content

API

Jump to bottom
ljleb edited this page Mar 6, 2023 · 37 revisions

The extension has 2 APIs:

  • external code API
  • web API

The external code API is useful when you want to control this extension from another extension.

The web API is useful when you want to communicate with the extension from a web client.

External Code API

The extension defines the external_code module. This module contains functions that you can use to control the extension for generating.

Examples

Create ControlNet Arguments

To create custom arguments and pass them to the extension for generating:

import importlib
external_code = importlib.import_module('extensions.sd-webui-controlnet.scripts.external_code', 'external_code')

def create_script_args(p: StableDiffusionProcessing):
    models = external_code.get_models()
    cn_units = [
        external_code.ControlNetUnit(
            model=models[0],  # assuming at least 1 model exists
            ...
        ),
        external_code.ControlNetUnit(
            model=models[1],  # assuming at least 2 models exist
            ...
        ),
        ...
    ]
    external_code.update_cn_script_in_processing(p, cn_units, is_img2img=isinstance(p, StableDiffusionProcessingImg2Img))

Update Existing Arguments

To update the ControlNet processing units from an existing script runner:

import importlib
external_code = importlib.import_module('extensions.sd-webui-controlnet.scripts.external_code', 'external_code')

def update_script_args(p: StableDiffusionProcessing):
    cn_units = external_code.get_all_units_in_processing(p)
    cn_units[0].resize_mode = external_code.ResizeMode.RESIZE
    cn_units[0].model = ...
    cn_units[1].image = None
    ...
    external_code.update_cn_script_in_processing(p, updated_units)

Remove All ControlNet Arguments

To remove all ControlNet processing units from an existing script runner: (essentially has the same effect as disabling the extension)

import importlib
external_code = importlib.import_module('extensions.sd-webui-controlnet.scripts.external_code', 'external_code')

def disable_controlnet(p: StableDiffusionProcessing):
    external_code.update_cn_script_in_processing(p, [])

Web API

The extension adds the following routes to the web API of the webui:

All routes use the Content-Type: application/json header.

The /controlnet/txt2img and /controlnet/img2img routes serve as a temporary fix. They will probably be marked as deprecated as soon as https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/8187 is merged into master of webui.

Route POST /controlnet/txt2img

This route can be used exactly like /sdapi/v1/txt2img. It accepts the same json properties, with the addition of an extra property "controlnet_units": []. This property accepts a list of ControlNetUnitRequest objects. Here's an example request body:

{
  "prompt": "a cinematic shot of an impressive ants war, ant melee, armageddon",
  "sampler_name": "Euler",
  "controlnet_units": [
    {
      "input_image": "base64...",
      "model": "diff_control_sd15_depth_fp16 [978ef0a1]"
    }
  ]
}

Route POST /controlnet/img2img

This route can be used exactly like /sdapi/v1/img2img. It accepts the same json properties, with the addition of an extra property "controlnet_units": []. This property accepts a list of ControlNetUnitRequest objects. Here's an example request body:

{
  "init_images": ["base64..."],
  "sampler_name": "Euler",
  "controlnet_units": [
    {
      "module": "depth",
      "model": "diff_control_sd15_depth_fp16 [978ef0a1]"
    }
  ]
}

Route GET /controlnet/model_list

Get the list of available ControlNet models. Returns a dictionary of the form {"model_list": [...]}. Each value of "model_list" is a valid candidate for the "model" property of the ControlNetUnitRequest object described below.

Route POST /controlnet/detect

Run a preprocessor by itself. Body of the route accepts a JSON object with the following property:

  • "controlnet_module" : preprocessor to use. defaults to "None"
  • "controlnet_input_images" : images to process. defaults to []
  • "controlnet_processor_res" : resolution of the preprocessor. defaults to 512
  • "controlnet_threshold_a" : first parameter of the preprocessor. only takes effect when preprocessor accepts arguments. defaults to 64
  • "controlnet_threshold_b" : second parameter of the preprocessor, same as "controlnet_threshold_a" for usage. defaults to 64

ControlNetUnitRequest JSON Object

This object describes a ControlNet processing unit entirely. It has the following properties:

  • "input_image" : image to use in this unit. defaults to ""
  • "module" : preprocessor to use on the image passed to this unit before using it for conditioning. defaults to "none"
  • "model" : name of the model to use for conditioning in this unit. accepts values returned by /controlnet/model_list route. defaults to "None"
  • "weight" : weight of this unit. defaults to 1
  • "resize_mode" : how to resize the input image so as to fit the output resolution of the generation. defaults to "Scale to Fit (Inner Fit)"
  • "lowvram" : whether to compensate low GPU memory with processing time. defaults to false
  • "processor_res" : resolution of the preprocessor. defaults to 64
  • "threshold_a" : first parameter of the preprocessor. only takes effect when preprocessor accepts arguments. defaults to 64
  • "threshold_b" : second parameter of the preprocessor, same as above for usage. defaults to 64
  • "guidance_start" : ratio of generation where this unit starts to have an effect. defaults to 0.0
  • "guidance_end" : ratio of generation where this unit stops to have an effect. defaults to 1.0
  • "guidance" : deprecated. same as "guidance_end" above. defaults to 1
  • "guessmode" : whether to infer and append a basic prompt for generation. defaults to true
Clone this wiki locally