What should i pick to fine-tune a model that is able for change the desgin of interior design

[abdelkareemkobo]

I want to create a model where I can provide an image of a room and instruct it to change the flooring, for example, from wood to parquet or another type of material. What should I choose?

[asomoza]

if you want the latest and the greatest and have a very high budget, it should be Flux by all means.

@sayakpaul did a really great job and an example with code for this, you can read about it here. You should be able to do what you want with just the correct dataset.

Just remember that that to use it commercially you'll need a license.

[asomoza]

you can use any model for what you want, the difference is in the quality of the final result and the training cost. If you want to be able to tell the model with words what you want you'll need to finetune it with the correct dataset, controlnet is not for instructions but to guide the generation with other condition images.

You can also do what you want without finetuning if you can use other images for it, for example you can use a mix between IP Adapters, img2img and controlnets for what you want, but you won't be able to tell written instructions to the model which was your original question.

[abdelkareemkobo]

Acutally i don't want to write prompts, I want to give it an image of the Tile type and make the model change the floor no need for text. what models do you recommend to start with if this what i want ?

[asomoza]

you can use https://huggingface.co/xinsir/controlnet-union-sdxl-1.0 for this, if you don't want to preserve the structure of the floor you can use a segmentation o normal map for the image (or if you want to preserve it a teed or depth) and also use the repaint one to change the type, if you want to control the color of it you can add it to img2img and pass it a color image with a high strength.

[abdelkareemkobo]

Sorry, but i don't understand most of what you said, i am new to computer vision, can you give me more references!

[asomoza]

oh, what you're trying to do is not something you can do if you don't understand what I said, so the best approach for you right now is to familiarize yourself with image generation, from the most basic knowledge to some more advanced concepts, what you're trying to do is the advanced concepts.

You can just search everything I said in something like google, but for now, I'll give you the links:

• controlnet union
• Image segmentation
• Normal maps
• Teed
• Depth
• image to image
• how to use controlnets

I'm sorry but I can't do more, I would love to make a guide about it or just guide you through it but I currently don't have the bandwidth for it.

In the future if I have time to do a guide or some example I will since this kind of work is something a lot of people have asked.