[HW Accel Support]: RK3328 (rock64) with '-rk' images #12577
Replies: 6 comments 23 replies
-
if privileged mode is enabled then the host is providing it, if it is not privileged then you must add to the docker command to map the right items in. the other questions I must relay to @MarcA711 |
Beta Was this translation helpful? Give feedback.
-
|
Hello, thank you for your answer, Does it imply that using the '-rk' image (and privileged mode) requires to have a BSP (or BSP-derived) kernel on the host ? |
Beta Was this translation helpful? Give feedback.
-
|
I checked the existence of device files on the host, inspired from this post: I find |
Beta Was this translation helpful? Give feedback.
-
|
Hey, I have not much knowledge about this soc (It is a somewhat older generation), but you might have two options:
maybe try to contact ayufan, he might be able to help you get a distro with all necessary drivers |
Beta Was this translation helpful? Give feedback.
-
|
Hi, thank you for the precious information. Somehow I suspected I had to get into kernel things... Learning everyday ^^ I guess the challenge is to find a kernel based on BSP (to get hardware specific drivers), and get it running on my Armbian.. Still few questions if I may... 1. Is there any specific versions to favor for the BSP kernel ?I saw people referring to versions 5.10 or 6.1, (maybe because they were the only ones available from BSP) but I don't know where did they get it from. With debian buster on Rock64 (Installed from Ayufan's OMV release), the apt command only reports other kernel versions (both older and more recent), either named after Ayufan, or not. 2. By reading forum, I don't understand how is the HW acceleration requested/initiated/used, with rkmpp, and eventually V4L2.Is V4L2 a new standard for accessing HW acceleration, and does that impact the usage of '-rk' image ? 3. Independently, I noticed that LibreElec provides images for Rock64 with recent kernels... Namely 5.10, 6.1, and newer...Once again I don't know whether these are based on BSP or not... Although I built it from source, I'm not ready to transfer this kernel to my Armbian (no initramfs, still requires learning)... If that's possible. I could not try with previous versions (kernel 6.1 and 5.10) because LE userspace is 32bit in these releases so Docker does not accept the Frigate image. 4. Last question, as pointed here, I should see specific devices in
|
Beta Was this translation helpful? Give feedback.
-
|
Hey, You can find some information for your board here: https://wiki.pine64.org/wiki/Mainline_Hardware_Decoding but these information can be outdated. AFAIK support for decoding (maybe also encoding) was mainlined and should be available when using a mainline kernel. Mainline kernel uses V4L2. However, you might need a forked version of ffmpeg (see fork below). I know the work was done by https://github.com/Kwiboo. Here is his kernel fork (but I guess most stuff is upstream): https://github.com/Kwiboo/linux-rockchip/tree/linuxtv-rkvdec-hevc-v2 and his FFmpeg fork: https://github.com/Kwiboo/FFmpeg Alternatively you can use the BSP kernel. This uses some drivers from rockchip and no v4l2 afaik. To access hardware from userspace, you need the https://github.com/rockchip-linux/mpp, or a version of ffmpeg that uses mpp, https://github.com/nyanmisaka/ffmpeg-rockchip (this is the version that Frigate uses). There are multiple versions of rockchip BSP kernels: 4.10, 4.19, 5.10, 6.1. Other version are probably mainline kernels.
The most up to date version that supports your soc. I am not sure if the current version 6.1 BSP (was released some months ago)
Yes, this depends on the kernel. If you use BSP kernel you need to use mpp (or FFmpeg+mpp). Mainline uses V4l2.
5.10 and 6.1 are likely BSP kernels. 6.6 is mainline. Maybe you could install LibreElec 6.1 kernel and download this static ffmpeg binary: https://github.com/MarcA711/Rockchip-FFmpeg-Builds/releases/tag/6.1-5 and check if it works. Usage instructions can be found here: https://github.com/nyanmisaka/ffmpeg-rockchip/wiki
This devices are only availabel if your kernel was built with rkvdec, rkvenc, ... etc drivers. So if these devices are not present, your kernel is likely mainline. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you so much !! Much clearer now. Also according to this, I guess I shouldn't try higher than 5.1 right ? (Rock64 soc is based on rk3328) |
Beta Was this translation helpful? Give feedback.
-
|
try using |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the suggestion, but LE does not have sudo: |
Beta Was this translation helpful? Give feedback.
-
|
ah ok tbh, I have no clue because I have no experience with the rk33XX socs. Your best option is to ask someone who has knowledge of this soc. Maybe you can contact ayufan. Sorry that I couldn't help you fix the issue! Edit: |
Beta Was this translation helpful? Give feedback.
-
|
Thank a lot for your help ! |
Beta Was this translation helpful? Give feedback.
-
|
Yeah would be great if you report back so I can improve the docs. Thank you. |
Beta Was this translation helpful? Give feedback.
-
|
I created issues in ffmpeg-rockchip and ayufan's repositories, and on LibreElec forum, we'll see how it goes :) |
Beta Was this translation helpful? Give feedback.
-
|
LibreElec seems to be based on mainline (see forum link)... Not sure yet if they make use of hardware acceleration. And as you pointed, information about mainline hw acceleration support is maybe not up to date: I don't really understand how it works though (once again ^^). So... new questions :) 1. Is V4L2 supposed to be the preferred way to access video devices in newer kernels ?Maybe Frigate (or ffmpeg) is planning to migrate to V4L2 in the future (if not available already) ? 2. Is this why we need to modify and recompile ffmpeg ?I guess we then need to pass custom parameters to ffmpeg in Frigate, correct ? I cannot find a lot of reference to v4l2 in Frigate documentation, could you point me to a guide or instructions so I can get started with it ? Thank you ! |
Beta Was this translation helpful? Give feedback.
-
|
Hey, I would do this: Try to get a distro with up to date mainline kernel (for example ayufans distro). Install ffmpeg from the repository and see if you can use hardware decoding (as explained here: https://wiki.pine64.org/wiki/Mainline_Hardware_Decoding#FFmpeg). If this won't work, you should try the FFmpeg fork of Jonas Karlman (he is the one who added support for the decoder in linux): https://github.com/Kwiboo/FFmpeg. You will need to compile from source. You will need to add You can also try to use a static "vanilla" ffmpeg from here https://github.com/BtbN/FFmpeg-Builds/releases and see if they support v4l2 requests by default. If this works you can probably use the default frigate image.
Yes, but usually end users don't need to think about this. You just add options to enable hw accel to ffmpeg for example
Frigate uses the default way to use hw acceleration on each platform. You can find the presets here https://github.com/blakeblackshear/frigate/blob/dev/frigate/ffmpeg_presets.py. If the option
Normally, you could just use vanilla ffmpeg, since it supports v4l2 I guess. But it seems that there is some patch (https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=2898) that is not yet merged in mainline ffmpeg, this is why you might need to compile the fork from jonas karlman.
Yes, usually most platform have their own options for hw acceleration, see the with ffmpeg presets that I linked above.
Here https://wiki.pine64.org/wiki/Mainline_Hardware_Decoding#FFmpeg it says, that you can use hw decoding with this command Moreover, you can read the comments in these patches: https://lore.kernel.org/linux-media/[email protected]/ and https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=2898 and look through these merges and their comments: FFmpeg/FFmpeg@master...Kwiboo:FFmpeg:v4l2request-2024-v2 |
Beta Was this translation helpful? Give feedback.
-
|
Awesome thanks a lot ! |
Beta Was this translation helpful? Give feedback.
-
|
Back with some news, but no success yet. I tried several ffmepg on 'regular' (non-BSP) kernels: 1. Current distribution ffmpeg (package manager)Ayufan Buster with 6.0.0 kernel, here is the log: 2. Prebuilt 'vanilla' ffmpegDownloaded from: 3. Libreelec 12 (kernel 6.10)Binary installed from their 'ffmpeg-tools' package, see here. So... no luck so far. I'm trying to compile Kwiboo's version (branch v4l2request-2024-v2) with "--enable-v4l2-request" but struggling with dependencies. Although I think I have everything, I still get this error message: It has very recent (yesterday) commits though so not sure it's compatible with my older kernel... I don't find documentation about building this specific fork. Let me know if you have some advices to share. |
Beta Was this translation helpful? Give feedback.
-
|
Still no success compiling ffmpeg with this flag... |
Beta Was this translation helpful? Give feedback.
-
|
It will still take some time. |
Beta Was this translation helpful? Give feedback.
-
|
Ok, that makes sense. Thanks again for your support, much appreciated ! |
Beta Was this translation helpful? Give feedback.
-
|
I'm following along here. I am looking to get video encoding on a 3328-based board. @fenouil-r Just wondering if you have a "working" setup yet? What kernel are you at? Should I set up 6.10 in the hopes that all will work in the coming months? |
Beta Was this translation helpful? Give feedback.
-
|
Hi, sorry for the delay I got busy with other projects recently. I did not manage to compile ffmpeg because of these dependencies problems. Not sure how/where to track these changes (integration of patches and new ffmpeg build release) though. |
Beta Was this translation helpful? Give feedback.
-
Describe the problem you are having
Hi,
My apologies if this topic seems recurring, but after reading 10's of page on various forums, I still don't have a clear idea on how it is supposed to work.
My current setup is a Rock64 running on Ayufan's OMV 5 distribution. Since then I updated kernel up to '6.0.0-1164-ayufan-ga2fd0c498'.
Now I discover Frigate and the amazing world of transcoding, detectors... and potential hardware acceleration :)
With 4 cameras (1920x1080) streams (via go2rtc), I am close to a constant 100% cpu just for ffmpeg... So I'm wondering if I could get the hw acceleration working on this board, with my limited knowledge (Spoiler: No).
For now I just tried to:
privileged: truein Portainer stackHere are some of the logs (same error for all cameras):
After getting a bash in the container and testing ffmpeg directly with:
I get:
This is where I started to read about hw acceleration on rockchip boards and realized I don't understand much about how this works :)
Here are my questions:
When using the docker image, should the host kernel provide some entries to the hw acceleration, or is this all handled from the container (hence privileged mode required) ?
What are the candidate kernels/distribution with such capability ?
I saw a lot of references to 5.10 and 6.1, but neither appear on my apt list...
From what I (poorly) understand, board vendors usually create kernels (bsp) exposing hw capabilities that end-up either integrated into mainline, or adapted to limited linux distributions (depending on community). Correct ?
Is it what Ayufan was providing for armbian on rockchip ?
I can't figure a timeline of what solutions were made available to access hw acceleration on this board (kernels, patches, scripts, ...), and what would be the optimal solution today...
Would it be possible to get a 'proper kernel' running with my current setup ?
If not, is there a debian distribution (I need OMV running) available for this board with the 'proper kernel' ?
That's already a lot, sorry... :)
Thank you for your help.
Version
0.14.0-a4eb435
Frigate config file
docker-compose file or Docker CLI command
Relevant log output
FFprobe output from your camera
Operating system
Debian
Install method
Docker Compose
Network connection
Wired
Camera make and model
Dafang (wize)
Any other information that may be helpful
No response
Beta Was this translation helpful? Give feedback.
All reactions