Stream a series of images on movement detection

[Stono]

Long story short i'm trying to use my ring cam to do some image detection to see if my cat has a rat in its mouth (at which point i'll lock/unlock the cat flap)! haha.

I wanted to leverage my ring cam, and this library gets me so close (thank you).

I successfully subscribe to the motion notification and get the last snapshot, but what i really need to do at that point is start a live stream and write an image out every say, 1 or 2 seconds, so i can run that through my model.

The example here:

ring/packages/examples/stream-example.ts

Line 27 in ea472dd

const call = await camera.streamVideo({

allows me to write an mp4 (interestingly btw the last segment is always corrupted).

I was reading a blog here about the fps=1 option, but can't get it to work:
https://www.bannerbear.com/blog/how-to-extract-images-from-a-video-using-ffmpeg/

I've tried a few combinations of flags and it just seems to fail silently. I've added debug: true to the Ring client but don't get any debug logs.

I'm no ffmpeg expert, so any help would really be appreciated, as i'm tantalisingly close to having a rodent free house.

[tsightler]

Assuming you really want images vs an mp4, which it sounds like you do, you probably want to transcode into png or jpg images. To do this you really don't need to do much more than set the frequency of how often you want to write an image file, you don't really need to use video filters for setting fps, etc, although that should work. I would suggest something simple like:

-r 1 -q:v 2 %02d.jpg

The -r option sets the rate, so, for example, 1 = 1 frame per second, while -r .5 would produce one image every 2 seconds and -r 2 would produce 2 images every second.

The -q:v 2 sets the quality of the jpeg image because the default is pretty compressed and not great, but you can also use some lossless format like png and then this isn't as important, but it is slower and takes more space. This post on superuser provides more details:

https://superuser.com/questions/135117/how-to-extract-one-frame-of-a-video-every-n-seconds-to-an-image

I know that method works because I previously used a similar approach in ring-mqtt to get snapshot images from battery cameras since they are unable to take snapshots while recording.

If ffmpeg is exiting and you don't know why you can include -report as an input option or set FFREPORT environment variable before running your code and it will write the output to a file so you can see why it is exiting.

[Stono]

Ahhh, thanks @tsightler.
Whilst i'm getting some output now, it doesn't appear to be working... i'm doing this:

    const stream = await camera.streamVideo({
      output: ['-r', '0.1', '-q:v', '2', path.join(process.cwd(), '%02d.jpg')]
    })

(i did try with -r 1 at first...). What that seems to be giving me is one .jpg per frame. On top of that, the jpg's don't appear to be jpg's!

I managed to get the ffmepg log working.
ffmpeg-20231113-075701.log

I used await camera.recordToFile('example.mp4', 5) to download a video first and parsed that with ffmpeg -i example.mp4 -r 1 output_%04d.png and that works, so just seems like i'm unable to do it from streamVideo.

[Stono]

Also fwiw now i have the logs i can see how the filter version didn't work: [vost#0:0/copy @ 0x144f0fc30] Filtergraph 'fps=1' was defined, but codec copy was selected. Filtering and streamcopy cannot be used together.

[Stono]

Ahhhh, got it! Had to set video: [], so:

    const stream = await camera.streamVideo({
      video: [],
      output: [
        '-r',
        '1',
        '-q:v',
        '2',
        path.join(process.cwd(), 'test/%02d.jpg')
      ]
    })

[tsightler]

Congrats! Hope it somehow manages to do what you want.

[Stono]

I did, here's the little rodent being captured.

Now to look for rats.
Thanks for your help (and the lib!).