Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using an image as visual prompt for generation #118

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

TheDenk
Copy link

@TheDenk TheDenk commented Feb 2, 2024

Hey :)

I've made some changes for using visual prompt image. It works nice with text prompt combinations and without them.
I tried to keep the style of writing code like yours.
Changes:

  1. Little changes in main pipelines.
  2. Code examples and image example were added in readme.
  3. Gradio demo was changed for load additional image.
  4. Additional file was added for infer with image prompt.

Examples:
visual_prompts_example

@TheDenk TheDenk changed the title added visual prompt code Using an image as visual prompt for generation Feb 2, 2024
@haofanwang
Copy link
Member

Interesting!

@haofanwang
Copy link
Member

I tested it locally and found that it might be also interesting to set the lighting direction as well.

截屏2024-02-05 02 45 01

But I think I would be better to add it as an individual pipeline_stable_diffusion_xl_instantid_img2img.py. We want to make the original pipeline as simple as possible. Thanks.

@TheDenk
Copy link
Author

TheDenk commented Feb 5, 2024

I separated pipelines and gradio apps to original and img2img.
All changes have been tested.

@haofanwang

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants