made a hacky script to rotoscope over video with stable diffusion

[Wax-Replica]

over the last couple days ive been playing with this script that uses ebsynth and stable diffusion to process frames of video semi-coherently.

the script is here

and i wrote a twitter thread with some discussion and a few examples here

[kungfooman]

Thank you for your effort of making so many example videos + sharing the script. I'm surprised how little Python code it is 👍

[Wax-Replica]

Thanks. Mostly I'm just rubbing two black boxes together to see if they stick, I'm not much of a programmer, but I thought it was fun enough to share

[cibernicola]

Seems interesting, I'm testing it:
Windows 10, some problems with paths

Path of ebsynth executable, filled
Input directory, filled.
Result:

I'll try to check deeper.

[Wax-Replica]

I can't quite tell what the problem is without seeing how the settings are set but I don't think it'll create a directory if it doesn't already exist, so make sure the output and scratch directory (where I have ebsynth putting the intermediate images) are set to an empty directory where it'll be able to write to as well. If that's not the problem I can't tell from this