Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor prompt parser #3535

Merged
merged 42 commits into from
Nov 12, 2024
Merged

Refactor prompt parser #3535

merged 42 commits into from
Nov 12, 2024

Conversation

AI-Casanova
Copy link
Collaborator

Full refactor of prompt parsing.

Moved prompt embeds inside of a class object that parses prompts at initialization and returns them when called.

Full scheduling capabilities restored

LRU cache for prompts, size adjustable from options

@vladmandic
Copy link
Owner

closes #2712

@vladmandic
Copy link
Owner

btw, i haven't commented on this for past few days because it looks ok, need to actually test it and i've been delaying that.

@vladmandic
Copy link
Owner

vladmandic commented Nov 11, 2024

@AI-Casanova 2 critical bugs in the first 2min of testing...

scenario: sdxl, simple prompt (one sentence), no batching, no scheduling

breaks with cfg 0-1 (no guidance)

RuntimeError: The size of tensor a (2048) must match the size of tensor b (4096) at non-singleton dimension 1

then i noticed that all images look washed out.
in general, there is something really off with guidance - see two grids with NEW vs OLD code:
image
image (1)

infotext for reproduction:

photo of a cute cyan robot with glowing glyph on a chest plate next to a road sign, next to a spaceship, walking sand with rocks and mountains and moon and planets visible in background, rich details, 4k
Steps: 15, Seed: 922182994, Sampler: UniPC, CFG scale: 2.0, Size: 1024x1024, Parser: Full, Backend: Diffusers, App: SD.Next, Version: e7f74c1, Operations: txt2img, Model: TempestV0.1-Artistic, Model hash: 9ca260b31e, Grid: 1x5x1, Pipeline: StableDiffusionXLPipeline, Script: XYZ Grid, X Type: [Param] Guidance scale, X Values: "2-10:5"

@AI-Casanova
Copy link
Collaborator Author

I have a sneaking suspicion that disabling the callback code will solve both issues.

vladmandic and others added 23 commits November 12, 2024 11:18
Signed-off-by: Vladimir Mandic <[email protected]>
Signed-off-by: Vladimir Mandic <[email protected]>
Signed-off-by: Vladimir Mandic <[email protected]>
Signed-off-by: Vladimir Mandic <[email protected]>
Signed-off-by: Vladimir Mandic <[email protected]>
Signed-off-by: Vladimir Mandic <[email protected]>
Signed-off-by: Vladimir Mandic <[email protected]>
Signed-off-by: Vladimir Mandic <[email protected]>
Signed-off-by: Vladimir Mandic <[email protected]>
Signed-off-by: Vladimir Mandic <[email protected]>
Signed-off-by: Vladimir Mandic <[email protected]>
Signed-off-by: Vladimir Mandic <[email protected]>
Signed-off-by: Vladimir Mandic <[email protected]>
Signed-off-by: Vladimir Mandic <[email protected]>
Signed-off-by: Vladimir Mandic <[email protected]>
Signed-off-by: Vladimir Mandic <[email protected]>
Signed-off-by: Vladimir Mandic <[email protected]>
Signed-off-by: Vladimir Mandic <[email protected]>
@vladmandic vladmandic merged commit 5200d33 into dev Nov 12, 2024
4 checks passed
@vladmandic vladmandic deleted the refactor-prompt branch November 12, 2024 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants