Thank you! Very nice interpolation

[jhogsett]

Hi, Thank You for your great video frame interpolation model!

I wanted to try it out, and compare it to VFIformer. I had previously created a web-based tool: "VFIformer-WebUI". It made sense to me to make a version of my tool that uses your model instead.

I've created "EMA-VFI-WebUI", a version of my application that uses your 2x interpolation model:

https://github.com/jhogsett/EMA-VFI-WebUI

It makes no changes to your code, and just needs to be overlaid on top. The readme file has details on how to install and run it. I think this will make it easy and fun to use your model and also compare it with VFIformer. My original application is at https://github.com/jhogsett/VFIformer-WebUI.

[GuozhenZhang1999]

Wow! This tool looks fantastic! Thank you so much for recognizing our work, and we'll definitely add a link to your tool in the README file. Personally, I was thinking that it might be even simpler and more efficient if we could compare the effectiveness of different models at the same time. Thanks again for sharing your awesome creation with us!

[jhogsett]

You're welcome! Thank you for your amazing research and model, and thanks for the link!

I have a keen interest in digitized film restoration. I'm very happy to find a model like yours.

I have started working on some comparisons myself, recreating some of my previous work with VFIformer. I'm happy to share some results here when I have them.

[Daniel-Kelvich]

@jhogsett I would be interested to see your comparison!

[GuozhenZhang1999]

You're welcome! Thank you for your amazing research and model, and thanks for the link!

I have a keen interest in digitized film restoration. I'm very happy to find a model like yours.

I have started working on some comparisons myself, recreating some of my previous work with VFIformer. I'm happy to share some results here when I have them.

Looking forward to seeing your progress! We will also provide a video that compares our method with a few others, and we hope it will be helpful to you!

[jhogsett]

@GuozhenZhang1999 @Daniel-Kelvich

Here's my initial test, which used to evaluate EMA-VFI. Background:

• My tool has a GIF-to-MP4 feature
• It increases the frame rate using frame interpolation, and increases the frame size using Real-ESRGAN
• Note: it uses simple binary interpolation (I plan to use your multi-interference model in a future version)

Original GIF:
https://user-images.githubusercontent.com/825994/220933026-3fe6bf25-8be7-490e-a90b-85a151e6b156.gif

VFIformer MP4:
https://user-images.githubusercontent.com/825994/224549158-7df76e49-28ec-4a3c-9b2b-4699729bd84f.mp4

EMA-VFI MP4:
https://user-images.githubusercontent.com/825994/224548062-4cad649c-5cdb-4f66-936d-e2296eb0fbc8.mp4

(Both examples have been created just now using the latest code. Rate increased 8X, size increased 4X)

What I observe:

• EMA-VFI is noticeably smoother, especially around 0:05

[GuozhenZhang1999]

Thank you for providing such a comprehensive example! Based on the results, it appears that our model indeed has better temporal consistency. If you are willing, please feel free to continue updating your progress on this post. Although we may not reply to each one individually, please know that we greatly appreciate your valuable contribution!

[jhogsett]

You're welcome @GuozhenZhang1999 ! I will post some more examples here for you and others to enjoy. I notice also that the EMA-VFI version looks slightly less dull than the VFIformer version.

[jhogsett]

Here's another similar example.

VFIformer: https://youtube.com/shorts/IZdEITUQx3U?feature=share

EMA-VFI: https://youtube.com/shorts/PLkOUxiFAuk?feature=share

I prefer the EMA-VFI version:

• The motion seems smoother and more fluid
• The image looks less dull than the VFIformer version
• The sudden disappearance of objects seems to be handled better (noticeable near the end)

Original GIF:

[jhogsett]

I've just posted a new YouTube video with a live comparison of EMA-VFI, VFIformer (footage inflated to 480 FPS) and the original footage (@ 30 FPS) https://youtube.com/shorts/n-YCv37bU9s?feature=share

[jhogsett]

Here is another video, same format. This time it's a police chase (footage inflated from 24 FPS to 768 FPS). https://youtube.com/shorts/En5VsJNVmnE?feature=share

[jhogsett]

Here is one more: https://youtube.com/shorts/P5TuFfLWIIM?feature=share

This is a carnival at night inflated from 15 FPS to 480 FPS (like inflating a 30 FPS video to 1000 FPS!).

EMA-VFI shows its quality very well, especially near the end, when the fast flashing lights seem to confuse VFIformer.

[jhogsett]

Three videos demonstrating the recovery of original video footage from time lapse videos using VFIformer and EMA-VFI:

Traffic: https://youtube.com/shorts/rwiyvQplQ28?feature=share

Carpenter: https://youtube.com/shorts/Emk3x-8jQNc?feature=share

Airplanes Landing: https://youtube.com/shorts/DRvn-a0u4zE?feature=share

[jhogsett]

The last of my video comparison series between VFIformer and EMA-VFI. For the final videos I chose very challenging content: ocean water, first-person POV, fast movement.

Electric Surfboard: https://www.youtube.com/shorts/BzBHsiEnNxg

Kite Surfer: https://www.youtube.com/shorts/W5otlVo989I

Rollercoaster: https://www.youtube.com/shorts/nb6uTRohPgU

[semel1]

If you can provide me with your original videos (preferably non-vertical) - I could make comparison using RIFE.

[jhogsett]

@semel1 each of the above YouTube videos includes a link to the royalty-free video used to create the comparison from MotionArray.com. Free watermarked preview versions can be downloaded without needing an account.

In particular I would be interested in seeing a version of this study, where the video has been inflated 32X to 768 FPS: https://www.youtube.com/shorts/En5VsJNVmnE

Edit: here's the link to the video: "The Pursuit Slaloms Left And Right" https://motionarray.com/stock-video/the-pursuit-slaloms-left-and-right-1171307/

[semel1]

@semel1 each of the above YouTube videos includes a link to the royalty-free video used to create the comparison from MotionArray.com. Free watermarked preview versions can be downloaded without needing an account.

In particular I would be interested in seeing a version of this study, where the video has been inflated 32X to 768 FPS: https://www.youtube.com/shorts/En5VsJNVmnE

Edit: here's the link to the video: "The Pursuit Slaloms Left And Right" https://motionarray.com/stock-video/the-pursuit-slaloms-left-and-right-1171307/

Please take a look at https://youtu.be/mFU8z-DSTds

[jhogsett]

Thank you. I'm seeing a significant loss of detail in your example, especially noticeable in the final part with the close-up of the car grill.

[2blackbar]

Hey man can you do one click nstaler with own venv so we dont have to deal with incompatible dependencies versions clashin?