Skip to content

Commit f9dae02

Browse files
author
kizamimi
committed
Add information about the optional items in the ReadME.
1 parent fcc0ae6 commit f9dae02

File tree

4 files changed

+130
-0
lines changed

4 files changed

+130
-0
lines changed

.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ result_lcm.png
1010
model.ckpt
1111
*.png
1212
!assets/img2img_example.png
13+
!assets/cfg_conparision.png
1314
!images/inputs/input.png
1415

1516
# Byte-compiled / optimized / DLL files

README-ja.md

+65
Original file line numberDiff line numberDiff line change
@@ -307,6 +307,71 @@ if __name__ == "__main__":
307307
run()
308308
```
309309

310+
## オプション
311+
312+
## Stochastic Similarity Filter
313+
314+
![demo](assets\demo_06.gif)
315+
316+
Stochastic Similarity Filterは動画入力時、前フレームからあまり変化しないときの変換処理を減らすことで、上のGIFの赤枠の様にGPUの負荷を軽減する。使用方法は以下のとおりである。
317+
318+
```
319+
stream = StreamDiffusion(
320+
pipe,
321+
[32, 45],
322+
torch_dtype=torch.float16,
323+
)
324+
stream.enable_similar_image_filter(similar_image_filter_threshold,similar_image_filter_max_skip_frame)
325+
```
326+
327+
関数で設定できる引数として以下がある。
328+
### similar_image_filter_threshold
329+
330+
- 処理を休止する前フレームと現フレームの類似度の閾値
331+
332+
### similar_image_filter_max_skip_frame
333+
334+
- 休止中に変換を再開する最大の間隔
335+
336+
## Residual CFG (RCFG)
337+
338+
![rcfg](assets\cfg_conparision.png)
339+
340+
RCFGはCFG使用しない場合と比較し、競争力のある計算量で近似的にCFGを実現させる方法である。StreamDiffusionの引数cfg_typeから指定可能である。
341+
342+
RCFGは二種類あり、negative promptの指定項目なしのRCFG Self-Negativeとnegative promptが指定可能なOnetime-Negativeが利用可能である。計算量はCFGなしの計算量をN、通常のCFGありの計算量を2Nとしたとき、RCFG Self-NegativeはN回で、Onetime-NegativeはN+1回で計算できる。
343+
344+
The usage is as follows:
345+
346+
```
347+
# CFG なし
348+
cfg_type = "none"
349+
350+
# 通常のCFG
351+
cfg_type = "full"
352+
353+
# RCFG Self-Negative
354+
cfg_type = "self"
355+
356+
# RCFG Onetime-Negative
357+
cfg_type = "initialize"
358+
359+
stream = StreamDiffusion(
360+
pipe,
361+
[32, 45],
362+
torch_dtype=torch.float16,
363+
cfg_type = cfg_type
364+
)
365+
366+
stream.prepare(
367+
prompt = "1girl, purple hair",
368+
guidance_scale = guidance_scale,
369+
delta = delta,
370+
)
371+
```
372+
373+
deltaはRCFGの効きをマイルドにする効果を持つ
374+
310375
# 開発チーム
311376

312377
[Aki](https://github.com/cumulo-autumn/),

README.md

+64
Original file line numberDiff line numberDiff line change
@@ -303,6 +303,70 @@ if __name__ == "__main__":
303303
run()
304304
```
305305

306+
## Optionals
307+
308+
## Stochastic Similarity Filter
309+
310+
![demo](assets\demo_06.gif)
311+
312+
Stochastic Similarity Filter reduces processing during video input by minimizing conversion operations when there is little change from the previous frame, thereby alleviating GPU processing load, as shown by the red frame in the above GIF. The usage is as follows:
313+
314+
```
315+
stream = StreamDiffusion(
316+
pipe,
317+
[32, 45],
318+
torch_dtype=torch.float16,
319+
)
320+
stream.enable_similar_image_filter(similar_image_filter_threshold,similar_image_filter_max_skip_frame)
321+
```
322+
323+
There are the following parameters that can be set as arguments in the function:
324+
325+
### similar_image_filter_threshold
326+
327+
- The threshold for similarity between the previous frame and the current frame before the processing is paused.
328+
329+
### similar_image_filter_max_skip_frame
330+
331+
- The maximum interval during the pause before resuming the conversion.
332+
333+
## Residual CFG (RCFG)
334+
335+
![rcfg](assets\cfg_conparision.png)
336+
337+
RCFG is a method for approximately realizing CFG with competitive computational complexity compared to cases where CFG is not used. It can be specified through the cfg_type argument in the StreamDiffusion. There are two types of RCFG: one with no specified items for negative prompts RCFG Self-Negative and one where negative prompts can be specified RCFG Onetime-Negative. In terms of computational complexity, denoting the complexity without CFG as N and the complexity with a regular CFG as 2N, RCFG Self-Negative can be computed in N steps, while RCFG Onetime-Negative can be computed in N+1 steps.
338+
339+
The usage is as follows:
340+
341+
```
342+
# w/0 CFG
343+
cfg_type = "none"
344+
345+
# CFG
346+
cfg_type = "full"
347+
348+
# RCFG Self-Negative
349+
cfg_type = "self"
350+
351+
# RCFG Onetime-Negative
352+
cfg_type = "initialize"
353+
354+
stream = StreamDiffusion(
355+
pipe,
356+
[32, 45],
357+
torch_dtype=torch.float16,
358+
cfg_type = cfg_type
359+
)
360+
361+
stream.prepare(
362+
prompt = "1girl, purple hair",
363+
guidance_scale = guidance_scale,
364+
delta = delta,
365+
)
366+
```
367+
368+
The delta has a moderating effect on the effectiveness of RCFG.
369+
306370
# Development Team
307371

308372
[Aki](https://github.com/cumulo-autumn/),

assets/cfg_conparision.png

3.78 MB
Loading

0 commit comments

Comments
 (0)