Skip to content

Latest commit

 

History

History
375 lines (308 loc) · 14.6 KB

README.md

File metadata and controls

375 lines (308 loc) · 14.6 KB
Raw

SD Webui Vectorscope CC

This is an Extension for the Automatic1111 Webui, which performs a kind of Offset Noise natively during inference, allowing you to adjust the brightness, contrast, and color of the generations.

Also supports both old & new Forge

Example Images


Base Image w/o Extension

Infotext
  • Checkpoint: realisticVisionV51
  • Positive Prompt: (high quality, best quality), a 4k cinematic photo of a gentleman in suit, street in a city at night, (depth of field, bokeh)
  • Negative Prompt: (low quality, worst quality:1.2), [EasyNegative, EasyNegativeV2]
Steps: 32, Sampler: DPM++ 2M Karras, CFG scale: 7.5, Seed: 3709157017, Size: 512x512, Denoising strength: 0.5
Clip skip: 2, Token merging ratio: 0.2, Token merging ratio hr: 0.2, RNG: CPU, NGMS: 4
Hires upscale: 2, Hires steps: 16, Hires upscaler: 2xNomosUni_esrgan_multijpg
Vibrant Cold "Movie when Mexico"
  • Alt: True
  • Saturation: 1.75
  • Noise: Ones
  • Scaling: 1 - Cos
  • Brightness: -5.0
  • Contrast: 2.5
  • Saturation: 0.75
  • R: -3.0
  • B: 3.0
  • Noise: Ones
  • Scaling: 1 - Sin
  • Brightness: 2.5
  • Contrast: -2.5
  • Saturation: 1.25
  • R: 1.5
  • G: 3.0
  • B: -4.0
  • Noise: Ones
  • Scaling: 1 - Sin

How to Use

Note: Since this Extension modifies the underlying latent tensor, the composition may change drastically depending on the parameters

Basic Parameters

  • Enable: Enable the Extension 💀

  • Alt: Cause the Extension effects to be stronger

    Technical Detail
    • This parameter makes the Extension modify the denoised Tensor instead of the x Tensor

  • Brightness, Contrast, Saturation: Adjust the overall brightness / contrast / saturation of the image

Color Channels

Channel Lower Higher
R Cyan Red
G Magenta Green
B Yellow Blue
  • The Extension also comes with a Color Wheel for visualization, which you can also click on to pick a color directly

The color picker isn't 100% accurate due to multiple layers of conversions...

Style Presets

  • To apply a Style, select from the Dropdown then click Apply Style
  • To save a Style, enter a name in the Textbox then click Save Style
  • To delete a Style, enter the name in the Textbox then click Delete Style
    • Style that was deleted is still in the styles.json in case you wish to retrieve it
  • Click Refresh Style to update the Dropdown if you edited the styles.json manually
You can also find pre-made Styles by the community available online
  • eg. The Photomatix Styles (right click on the link, click Save link as, then save the .json file into the sd-webui-vectorscope-cc extension folder)

Advanced Parameters

  • Process Hires. fix: Enable this option to process during the Hires. fix phase too
    • By default, this Extension only functions during the regular phase of the txt2img mode
  • Process ADetailer: Enable this option to process during the ADetailer phase too
    • Will usually cause a square of inconsistent colors
  • Randomize using Seed: Enable this option to use the current generation seed to randomize the basic parameters
    • Randomized results will be printed in the console

Noise Settings

let x denote the latent Tensor ; let y denote the operations

  • Straight: All operations are calculated on the same Tensor
    • x += x * y
  • Cross: All operations are calculated on the Tensor opposite of the Alt. setting
    • x += x' * y
  • Ones: All operations are calculated on a Tensor filled with ones
    • x += 1 * y
  • N.Random: All operations are calculated on a Tensor filled with random values in normal distribution
    • x += randn() * y
  • U.Random: All operations are calculated on a Tensor filled with random values in uniform distribution
    • x += rand() * y
  • Multi-Res: All operations are calculated on a Tensor generated with multi-res noise algorithm
    • x += multires() * y
  • Abs: Calculate using the absolute values of the chosen Tensors instead
    • x += abs(F) * y

Infotext
  • Checkpoint: realisticVisionV51
  • Positive Prompt: (high quality, best quality), a 4k photo of a cute dog running in the snow, mountains, day, (depth of field, bokeh)
  • Negative Prompt: (low quality, worst quality:1.2), [EasyNegative, EasyNegativeV2]
  • Brightness: 2.5
  • Contrast: 2.5
  • Alt: True
  • Scaling: 1 - Cos
Steps: 24, Sampler: DPM++ 2M Karras, CFG scale: 7.5, Seed: 1257068736, Size: 512x512, Denoising strength: 0.5
Clip skip: 2, Token merging ratio: 0.2, Token merging ratio hr: 0.2, RNG: CPU, NGMS: 4
Hires upscale: 1.5, Hires steps: 16, Hires upscaler: 2xNomosUni_esrgan_multijpg

Scaling Settings

By default, this Extension offsets the noise by the same amount every step. But depending on the Sampler and Scheduler used, and whether Alt. was enabled or not, the effects might be too strong during the early or the later phase of the process, which in turn causes artifacts.

  • Flat: Default behavior
  • Cos: Cosine scaling (High -> Low)
  • Sin: Sine scaling (Low -> High)
  • 1 - Cos: (Low -> High)
  • 1 - Sin: (High -> Low)

Infotext
  • Checkpoint: realisticVisionV51
  • Positive Prompt: (high quality, best quality), a 4k photo of a cute cat standing at a flower field in a park, day, (depth of field, bokeh)
  • Negative Prompt: (low quality, worst quality:1.2), [EasyNegative, EasyNegativeV2]
  • Alt: True
  • Noise: Straight Abs.
Steps: 24, Sampler: DPM++ 2M Karras, CFG scale: 7.5, Seed: 3515074713, Size: 512x512, Denoising strength: 0.5
Clip skip: 2, Token merging ratio: 0.2, Token merging ratio hr: 0.2, RNG: CPU, NGMS: 4
Hires upscale: 1.5, Hires steps: 12, Hires upscaler: 2xNomosUni_esrgan_multijpg

Buttons

  • Reset: Reset all Basic and Advanced parameters to the default values
  • Randomize: Randomize the Brightness, Contrast, Saturation, R, G, B parameters

Settings

The following settings are in the Vectorscope CC section under the Stable Diffusion category of the Settings tab

  • Append the parameters to the infotext
  • Disable do_not_save_to_config to use the Webui Defaults functionality
  • Set the minimum and maximum range for each parameter

Roadmap

  • Extension Released!
  • Add Support for X/Y/Z Plot
  • Implement different Noise functions
  • Implement Randomize button
  • Implement Style Presets
  • Implement Color Wheel & Color Picker
  • Implement better scaling algorithms
  • Add API Docs
  • Append Parameters to Infotext
  • Improved Infotext Support (by. catboxanon)
  • Add HDR Script
  • Add Support for SDXL
  • Implement Gradient features

API

You can use this Extension via API by adding an entry to the alwayson_scripts of your payload. An example is provided. The args are sent in the following order in an array:

Parameter Type
Enable bool
Alt. bool
Brightness float
Contrast float
Saturation float
R float
G float
B float
Hires. fix bool
ADetailer bool
Randomize bool
Noise Method str
Scaling str

Known Issues

  • In rare occasions, this Extension has little effects when used with certain LoRAs
  • Works better / worse with certain Samplers

HDR

Discussion Thread

In the Script Dropdown at the bottom, there is now a new High Dynamic Range option:

  • This script will generate multiple images ("Brackets") of varying brightness, then merge them into 1 HDR image
  • (Recommended) Use a deterministic sampler and high enough steps. Euler (not Euler a) works well in my experience

Options

  • Brackets: The numer of images to generate
  • Gaps: The brightness difference between each image
  • Automatically Merge: When enabled, this will merge the images using an OpenCV algorithm and save to the HDR folder in the outputs folder
    • Disable this if you want to merge them yourself using better external program
Offset Noise TL;DR

The most common version of Offset Noise you may have heard of is from this blog post, where it was discovered that the noise functions used during training were flawed, causing Stable Diffusion to always generate images with an average of 0.5 (ie. grey).

ie. Even if you prompt for dark/night or bright/snow, the average of the image is still "grey"

Technical Explanations

However, this Extension instead tries to offset the latent noise during the inference phase. Therefore, you do not need to use models that were specially trained, as this can work on any model.

How does this work?

After reading through and messing around with the code, I found out that it is possible to directly modify the Tensors representing the latent noise used by the Stable Diffusion process.

The dimensions of the Tensors is (X, 4, H / 8, W / 8), which represents X batch of noise images, with 4 channels, each with (W / 8) x (H / 8) values

eg. Generating a single 512x768 image will create a Tensor of size (1, 4, 96, 64)

Then, I tried to play around with the values of each channel and ended up discovering these relationships. Essentially, the 4 channels correspond to the CMYK color format for SD1 (Y'CbCr for SDXL), hence why you can control the brightness as well as the colors.

Vectorscope?

The Extension is named this way because the color interactions remind me of the Vectorscope found in Premiere Pro's Lumetri Color. Those who are experienced in Color Correction should be rather familiar with this Extension.

Yes. I'm aware that it's just how digital colors work in general.

We've come full circle (*ba dum tss) now that a Color Wheel is actually added.