Model download

This page documents multiple sources of models for the ControlNet extension.

Introduction

ControlNets allow yout to guide the image generation with user supplied images. The original implentation with examples can be found in https://github.com/lllyasviel/ControlNet.

Installation

After you download models, please put them under models/ControlNet. You need to download model files ending with ".pth" / ".safetensors" / ".bin". Do not right-click the filenames in HuggingFace website to download. Some users right-clicked those HuggingFace HTML websites and saved those HTML pages as PTH/YAML files. They are not downloading correct files. Instead, please click the small download arrow “↓” icon in HuggingFace to download.

Stable Diffusion 1.5 / 2.0

Official ControlNet 1.1 models

Stable Diffusion 1.5 and Stable Diffusion 2.0 ControlNet models are compatible with each other. There are three different type of models available of which one needs to be present for ControlNets to function

• LARGE - these are the original models supplied by the author of ControlNet. Each of them is 1.45 GB large and can be found here.
• MEDIUM - these have been shrunk to half their size by converting them to 16 bit. Each of them is 723 MB large and can be found here.
• SMALL - these are LoRa implementations that only use 136 MB each and can be found here.

ControlNet++ models

ControlNet++ offers better alignment of output against input condition by replacing the latent space loss function with pixel space cross entropy loss between input control condition and control condition extracted from diffusion output during training. You can download models in following sources:

• LARGE (fp32) - https://huggingface.co/spaces/limingcv/ControlNet-Plus-Plus/tree/main/checkpoints
• MEDIUM (fp16) - https://huggingface.co/huchenlei/ControlNet_plus_plus_collection_fp16/tree/main

Stable Diffusion XL

There is no official SDXL ControlNet model. All models come from Stable Diffusion community. Here are some collections of SDXL models:

Lvmin's collection

• diffusers ControlNet
• Controllllite
• Stability AI Control LoRA
• T2I Adapter

Qinglong's Controllllite SDXL models

• Normal Dsine(NEW)
• Tile realistic
• Tile anime
• MLSD
• DW pose
• Normal
• Recolor Luminance
• Softedge
• Segment animeface

TTPLanet Tile Realistic

A realistic tile model trained by community for SDXL.

2vXpSwA7 cnlllite

grey2color and anystyle

MistoLine

mistoLine SDXL softedge model

Kataragi's SDXL models

Kataragi has several SDXL models trained. Half precision models (2.5G) and rank128 LoRAs (400MB) are provided.

• Inpaint
• Recolor
• Lineart
• Line2Color

SDXL lineart model that specializes in anime

Extensions

Other models supported in the ControlNet extension.

Animal Openpose [SD1.5] Original Project repo - Models

IPAdapter [SD1.5 / SDXL] Original Project repo - Models

IPAdapter FaceID [SD1.5 / SDXL] Original Project repo - Models

IPAdapter Composition [SD1.5 / SDXL] Models [Note: need to rename model files to ip-adapter_plus_composition_sd15.safetensors and ip-adapter_plus_composition_sdxl.safetensors]

PhotoMaker [SDXL] Original Project repo - Models

InstantID [SDXL] Original Project repo - Follow instruction in here

Fooocus Inpaint [SDXL] patch - Needs a little more explanation

T2I Adapter

https://huggingface.co/TencentARC/T2I-Adapter/tree/main/models