How to enable hardware H.264 encoding (h264_v4l2m2m)?

[Jachimo]

I've got a birdhouse cam set up with a Raspberry Pi Zero W and the original Pi Zero camera (5MP OV5647). I realize this is all relatively old hardware, but several years ago I had it working and it worked beautifully. So I decided to set it up again, and in the process installed Raspbian Bullseye… and things have not gone well since.

Anyway, at this point I have abandoned Motion for MotionPlus (I figured if I was going to invest the time in debugging, it might as well be on the 'new' software) and I've got it talking to the camera via the new libcamera API, and recognizing motion / triggering recordings.

However, while it will get frames from the camera and check for motion at a reasonable rate (I'm aiming for 960x720 @ 10FPS, nothing crazy), it's utterly incapable of actually outputting a movie at more than a handful of FPS. Although the actual framerate varies a bit, most recorded MP4 files have framerates around 1.1-1.2 FPS.

Since the system seems able to stream video at ~8 FPS, I suspect the issue is with re-encoding the video to H.264. And it appears that the default H.264 codec that gets used when you select movie_container mp4 is libx264, running completely on the CPU. Presumably with motion detection taking up 40-60% CPU already, there's just not enough cycles left to compress the output in real-time.

To try and force use of the hardware encoder, I changed the movie_container line to movie_container mp4:h264_v4l2m2m, but this immediately throws an error "Encoder requires yuv420p pixel format" that I'm not sure how to work around. It doesn't appear that MotionPlus has the same "palette" options that Motion has/had, in terms of forcing specific pixel formats... and I can't figure out if there's some way to convert whatever internal representation MotionPlus uses to yuv420p in preparation for the hardware encoder.

Can anyone suggest how to get hardware H.264 encoding working? Or is there another output format that's easier on the hardware to write to disk? I have a fairly big (128GB) MicroSD card in the PiZeroW, so I am not especially space-constrained. But I can't simply switch to a more powerful Raspberry Pi. The Pi Zero W and its camera are basically built-in to the birdhouse at this point, and the wiring for its power supply likely won't support a full-size Pi, even if I could obtain one and figure out how to install it. So throwing more hardware at the problem is not really what I am looking for.

I know that this hardware previously produced much better results, and the capability for HW H.264 encoding is there on the board, so I'm really just hoping there's a way to get back to that high-water mark.

MotionPlus Configuration File

daemon off
setup_mode off
log_level 6
log_type ALL
target_dir /var/motion
emulate_motion off
threshold 175
minimum_motion_frames 1
event_gap 60
pre_capture 3
post_capture 10
movie_output on
movie_max_time 300
movie_quality 45
#movie_container mp4:h264_v4l2m2m
movie_container mp4
movie_filename %Y%m%dT%H%M%S
webcontrol_port 8080
webcontrol_localhost on
webcontrol_parms 2
stream_preview_scale 25
stream_preview_method mjpg
config_dir /etc/motionplus/devices

Camera Configuration File

device_name birdhouse
device_id 1
libcam_device camera0
width 960
height 720
framerate 5
text_right %Y-%m-%d\n%T
text_left %{fps}\n%D/%o px
text_changes on

[Mr-Dave]

I do not recall any difference between Motion and MotionPlus with the pixel formats for the encoding. You may want to quickly just try to set up Motion using the libcamerify motion method and validate that the performance and functionality is substantially different.

The next option would be to use the extpipe option of either Motion or MotionPlus and then just specify the HW and encoding options for ffmpeg directly into the command used to encode. H264/5 do use the most resources. The Pi's typically used the omx encoder but I've not tested that recently to see if it is still in the latest version of RaspberryPiOS.

[Jachimo]

You may want to quickly just try to set up Motion using the libcamerify motion method and validate that the performance and functionality is substantially different.

I tried using classic Motion instead of MotionPlus and the performance is basically the same, around 1 FPS in the final mp4 file.

If you have any thoughts on how to enable hardware encoding with Motion, I'd be interested.

However, I am fairly confident that the OMX encoder is gone. I guess that I would need to use h264_v4l2m2m instead as well?

[Mr-Dave]

You may want to double check omx. I see it as "--enable-omx" in the ffmpeg output as well as in the codec options on my bullseye version.

DEV.LS h264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_v4l2m2m ) (encoders: libx264 libx264rgb h264_omx h264_v4l2m2m h264_vaapi )

--enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2

[Jachimo]

If I go back to classic Motion instead of MotionPlus, and invoke it as libcamerify motion, the first thing I noticed was an error in the logs that didn't happen with MotionPlus: [1:ml1] [ERR] [VID] v4l2_fps_set: Error setting fps. Return code -1. I've Googled that message but can't find a clear cause or solution. (And I get it even if I comment the framerate line in motion.conf out completely, so it doesn't even like the default value.) So that's a bit odd—not sure what to make of that. I am running Motion with debug logging enabled, but it doesn't list a bunch of valid framerates or anything that I can see. However, it looks like things still continue to work after that error is thrown, so I ignored it for the moment.

Beyond that, if I set movie_codec mp4:h264_omx, I get a bunch of log messages like this:

motion[19380]: [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: libOMX_Core.so not found
motion[19380]: [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: libOmxCore.so not found

This is with the version of Motion from the .deb release file (reports as version 4.5.1). I'm not sure what to do, and my belief that it stemmed from the OMX codec being removed in stock Bullseye comes from this page:
https://forums.raspberrypi.com/viewtopic.php?t=323351

I also tried setting movie_codec mp4:h264_v4l2m2m in motion.conf. With this configuration, I get the following errors:

motion[19473]: [1:ml1] [NTC] [ENC] ffmpeg_set_codec_preferred: Using codec h264_v4l2m2m
motion[19473]: [1:ml1] [INF] [ENC] ffmpeg_set_quality: h264_v4l2m2m codec vbr/crf/bit_rate: 2673000
motion[19473]: [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: Encoder requires yuv420p pixel format.
motion[19473]: [1:ml1] [INF] [ENC] Above message repeats 19 times
motion[19473]: [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: output VIDIOC_REQBUFS failed: Cannot allocate memory
motion[19473]: [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: no v4l2 output context's buffers
motion[19473]: [1:ml1] [INF] [ENC] ffmpeg_avcodec_log: can't configure encoder

Based on the docs, yuv420p format is palette 17, which is also the default... so I'm not sure why I'd have to set it manually, but I tried it just for thoroughness and got exactly the same result.

[Mr-Dave]

OK. That is a bit disappointing that they did that.

Your options....

1. You could try to compile with a revision to the encoding. In the prior variants of ffmpeg, the v4l2m2m required a different format so, we set it here. As you can see we did a special pixel format and now based upon the error message you've shown, they seem to want the different one.....sigh..
2. Skip the recompiling and instead focus upon creating a ffmpeg command line that invokes the HW encoding. This would be adding revisions to the sample line here (probably the last sample) and then adding in a parameter such as -hwaccel h264_v4l2m2m

[Mr-Dave]

FYI. This is now embedded into the master branch. From my testing, it makes a substantial difference. Essentially no increase in CPU was noted when recording a video.