Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarify LTXVImgToVideo's seeding for image_noise_scale #6214

Open
rc-gr opened this issue Dec 25, 2024 · 0 comments
Open

Clarify LTXVImgToVideo's seeding for image_noise_scale #6214

rc-gr opened this issue Dec 25, 2024 · 0 comments
Labels
Feature A new feature to add to ComfyUI.

Comments

@rc-gr
Copy link

rc-gr commented Dec 25, 2024

Feature Idea

Currently, it is unclear how seeding is done when using the LTXVImgToVideo node with image_noise_scale > 0.0, which produces unexpected results in the final generation.

For example, using your LTXV I2V workflow:
LTXVNoiseIssue.zip

Using the enclosed files as examples, image 2 was generated using the exact workflow and seed as image 1. But because image_noise_scale is > 0.0, there is some variance in the results that is neither immediately obvious nor reflected in the workflow. In this example, the cat in (1) appears to be almost squinting its eyes by the end. But in (2), the cat shuts its eyes and even lowers its head.

You may refer to the following issue for more details, using a different workflow but with the same underlying node, and even more variance in the results: mcmonkeyprojects/SwarmUI#491 (comment)

On that note, would it be possible to implement something like SamplerCustom's noise_seed and control_after_generate to give more control and provide more clarity on how image_noise_scale is being used?

Personally, at least from how I've been using it, having some noise is still desirable for LTXV to promote more motion in the final output for some images. But the randomness that goes with it when it's neither specified nor expected reduces the reliability, which in turn discourages its use.

Existing Solutions

None. The alternative would be to zero out image_noise_scale to eliminate the randomness altogether, which may not be ideal in some cases.

Other

mcmonkeyprojects/SwarmUI#491 (comment)

@rc-gr rc-gr added the Feature A new feature to add to ComfyUI. label Dec 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature A new feature to add to ComfyUI.
Projects
None yet
Development

No branches or pull requests

1 participant