Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prep HEVC decoder for upstreaming #6519

Draft
wants to merge 784 commits into
base: rpi-6.12.y
Choose a base branch
from

Conversation

6by9
Copy link
Contributor

@6by9 6by9 commented Dec 6, 2024

Adds the driver with the multiplanar SAND format support, and then adds the minimal patch to reintroduce the legacy single planar format.

Working with both stock FFmpeg, and John's new multi-planar branch https://github.com/jc-kynesim/rpi-ffmpeg/tree/dev/5.1.6/sandm_1

Mesa does still need updating to support the SAND height of 0.

@popcornmix popcornmix force-pushed the rpi-6.12.y branch 2 times, most recently from 1348250 to 2b062ea Compare December 10, 2024 14:47
@6by9 6by9 force-pushed the rpi-6.12.y-hevc_dec branch from 3406809 to 6168cbb Compare December 12, 2024 17:36
@popcornmix popcornmix force-pushed the rpi-6.12.y branch 2 times, most recently from 2accb17 to 49679ed Compare December 20, 2024 13:36
pelwell and others added 24 commits January 10, 2025 11:32
In order to avoid losing residue bytes when a receive is terminated
early, set the destination width to single bytes.

Link: raspberrypi#6365

Signed-off-by: Phil Elwell <[email protected]>
The xHC may commence Host Initiated Data Moves for streaming endpoints -
see USB3.2 spec s8.12.1.4.2.4. However, this behaviour is typically
counterproductive as the submission of UAS URBs in {Status, Data,
Command} order and 1 outstanding IO per stream ID means the device never
enters Move Data after a HIMD for Status or Data stages with the same
stream ID. For OUT transfers this is especially inefficient as the host
will start transmitting multiple bulk packets as a burst, all of which
get NAKed by the device - wasting bandwidth.

Also, some buggy UAS adapters don't properly handle the EP flow control
state this creates - e.g. RTL9210.

Set Host Initiated Data Move Disable to always defer stream selection to
the device. xHC implementations may treat this field as "don't care,
forced to 1" anyway - xHCI 1.2 s4.12.1.

Signed-off-by: Jonathan Bell <[email protected]>
The default link frequency of 450MHz has been noted to interfere
with GPS if they are in close proximty.
Add the option for 453 and 456MHz to move the signal slightly out
of the band. (447MHz can not be offered as corruption is then observed
on the 133x992 10bit mode).

Signed-off-by: Dave Stevenson <[email protected]>

fixup imx477 gps
Copy of the imx708 change.

Signed-off-by: Dave Stevenson <[email protected]>
Attempting to start a non-idle channel causes an error message to be
logged, and is inefficient. Test for emptiness of the desc_issued list
before doing so.

Signed-off-by: Phil Elwell <[email protected]>
The Raspberry Pi RP1 includes 2 M3 cores running firmware. This driver
adds a mailbox communication channel to them via a doorbell and some
shared memory.

Signed-off-by: Phil Elwell <[email protected]>
The RP1 firmware runs a simple communications channel over some shared
memory and a mailbox. This driver provides access to that channel.

Signed-off-by: Phil Elwell <[email protected]>
Declare the communications channel to RP1.

Signed-off-by: Phil Elwell <[email protected]>
Provide remote access to the PIO hardware in RP1. There is a single
instance, with 4 state machines.

Signed-off-by: Phil Elwell <[email protected]>
Declare the device that proxies RP1's PIO hardware.

Signed-off-by: Phil Elwell <[email protected]>
Use the PIO hardware on RP1 to implement a PWM interface.

Signed-off-by: Phil Elwell <[email protected]>
Enable building of the pwm-pio-rp1 driver, which is Pi 5-specific.

Signed-off-by: Phil Elwell <[email protected]>
Add an overlay to enable a single-channel PIO-assisted PWM interface on any
header pin.

Signed-off-by: Phil Elwell <[email protected]>
This is a SPI to powerline chipset with host-side Ethernet interface.
Is is usually used in e-mobility environments, e.g. on
Electrical Vehicle Supply Equipment (EVSE) side.

Signed-off-by: Michael Heimpold <[email protected]>
The documentation isn't very clear explaining how to enable SPI CS
active-high and it takes a long time to understand it. Adding a specific
overlay as a simple example on how to invert this signal can help
understand the solution.

Link: https://forums.raspberrypi.com/viewtopic.php?t=378222
Signed-off-by: Iker Pedrosa <[email protected]>
If autonomous speed negotiation is unreliable then brcm_pcie_set_gen()
can be used to override/limit this behaviour. However, setting the limit
after the linkup poll means the link can temporarily enter a bad speed
which may result in link failure. Move the speed setup to just prior to
releasing perst_n.

Fixes: 0693b42 ("PCI: brcmstb: Split post-link up initialization to brcm_pcie_start_link()")

Signed-off-by: Jonathan Bell <[email protected]>
Using increased bit depth for no reason increases power
consumption, and differs from the behaviour prior to the
conversion to use the HDMI helper functions.

Initialise the state max_bpc and requested_max_bpc to the
minimum value supported. This only affects Raspberry Pi,
as the other users of the helpers (rockchip/inno_hdmi and
sunx4i) only support a bit depth of 8.

Signed-off-by: Dave Stevenson <[email protected]>
If an infoframe was ever enabled, duplicate_state would
memcpy the infoframe including the "set" flag. The
infoframe functions then never cleared it, so once set
it was always set. This was most obvious with the HDR
infoframe as it resulted in bad colour rendering.

Signed-off-by: Dave Stevenson <[email protected]>
Drop from RGB to YUV422 output if RGB couldn't be supported
within the defined max_bpc and TMDS rates, and then try
dropping max_bpc.

Signed-off-by: Dave Stevenson <[email protected]>
Signed-off-by: Giedrius Trainavičius <[email protected]>
marcone and others added 5 commits January 10, 2025 13:21
commit c13094b upstream.

on 32-bit kernels, iomap_write_delalloc_scan() was inadvertently using a
32-bit position due to folio_next_index() returning an unsigned long.
This could lead to an infinite loop when writing to an xfs filesystem.

Signed-off-by: Marco Nelissen <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Reviewed-by: Darrick J. Wong <[email protected]>
Reviewed-by: Christoph Hellwig <[email protected]>
Signed-off-by: Christian Brauner <[email protected]>
The DT property arm,cpu-registers-not-fw-configured tells the kernel
that the ARM architectural timer has not been configured by the
firmware. This prevents the use of a vDSO - a faster alternative to a
syscall for some common kernel operations.

However, on Pi 4 the firmware does configure the timer, so this property
is unnecessary. Delete it.

Signed-off-by: Phil Elwell <[email protected]>
Make sure the sdhost driver doesn't use requests bigger than SWIOTLB
can handle.

Copied from [1].

Link: raspberrypi#6589
Signed-off-by: Phil Elwell <[email protected]>
[1] d4dd9bc ("mmc: bcm2835: Take SWIOTLB memory size limitation
into account")
We can avoid calling the v3d_clock_up_put and v3d_clock_up_get
when a job is submitted to a CPU queue. We don't need to change
the V3D core frequency to run a CPU job as it is executed on
the CPU. This way we avoid delaying timestamps CPU jobs by 4.5ms
that is the time that it takes the firmware to increase the V3D
core frequency.

Fixes: fe6a858 ("drm/v3d: Correct clock settng calls to new APIs")
Signed-off-by: Jose Maria Casanova Crespo <[email protected]>
Reviewed-by: Maíra Canal <[email protected]>
Prior to [1], an fb_ops member of 0 was intepreted as a request for a
default value. This saves source code but requires special handling by
the framework, slowing down all accesses for no runtime benefit.

Use the new __FB_DEFAULT_ macros to explicitly select default handlers
in the bcm2708_fb driver. Also remove the pointless wrappers around
cfb_fillrect and cfb_imageblit - call them directly.

Link: https://forums.raspberrypi.com/viewtopic.php?p=2286016#p2286016
Signed-off-by: Phil Elwell <[email protected]>
[1] 8813e86 ("fbdev: Remove default file-I/O implementations")
@6by9 6by9 force-pushed the rpi-6.12.y-hevc_dec branch from 6168cbb to 310d12e Compare January 14, 2025 16:52
pelwell and others added 21 commits January 14, 2025 16:55
Now that the upstream driver is overclockable, switch to using it in
preference of the downstream driver (which can be deleted by a followup
commit).

Signed-off-by: Phil Elwell <[email protected]>
The principal differences between the downstream SDHOST driver and the
version accepted upstream driver are that the upstream version loses the
overclock support and DMA configuration via DT, but gains some tidying
up (and maintenance by the upstream devs).

Add the missing features (with the exception of the low-overhead logging)
as a patch to the upstream driver.

Signed-off-by: Phil Elwell <[email protected]>
This reverts commit fad2884.

We are about to replace this with manual_complete
By default when the last request object is completed, the whole
request completes as well.

But sometimes you want to manually complete a request in a driver,
so add a manual complete mode for this.

In req_queue the driver marks the request for manual completion by
calling media_request_mark_manual_completion, and when the driver
wants to manually complete the request it calls
media_request_manual_complete().

Signed-off-by: Hans Verkuil <[email protected]>
Manually complete the requests: this tests the manual completion
code.

Signed-off-by: Hans Verkuil <[email protected]>
Keep track of the number of requests and request objects of a media
device. Helps to verify that all request-related memory is freed.

Signed-off-by: Hans Verkuil <[email protected]>
Adds a binding for the HEVC decoder found on the BCM2711 / Raspberry Pi 4,
and BCM2712 / Raspberry Pi 5.

Signed-off-by: Dave Stevenson <[email protected]>
The Raspberry Pi HEVC decoder uses a tiled format based on
columns for 8 and 10 bit YUV images, so document them as
NV12MT_COL128 and NV12MT_10_COL128.

Signed-off-by: Dave Stevenson <[email protected]>
Add V4L2_PIXFMT_NV12MT_COL128 and V4L2_PIXFMT_NV12MT_10_COL128
to describe the Raspberry Pi HEVC decoder NV12 multiplanar formats.

Signed-off-by: Dave Stevenson <[email protected]>
The BCM2711 and BCM2712 SoCs used on Rapsberry Pi 4 and Raspberry
Pi 5 boards include an HEVC decoder block. Add a driver for it.

Signed-off-by: John Cox <[email protected]>
Signed-off-by: Dave Stevenson <[email protected]>
Adds the relevant node for the HEVC decoder on BCM2711 / Pi4.

Signed-off-by: Dave Stevenson <[email protected]>
The SAND handling had been using what was believed to be a runtime
parameter in the modifier, however that has been clarified that
all permitted variants of the modifier must be advertised, so
making it variable wasn't practical.

With a rationalisation of how the producers of this format are
configured, we can switch to a variant that doesn't have as much
variation, and can be configured such that only 2 options are
required.

Add a modifier with value 0 to denote that the height of the luma
column matches the buffer height, and chroma column will be half
that due to YUV420.
A modifier of 1 denotes that the height of the luma column still
matches the buffer height, but the chroma column height is the same.
This can be used to replicate the previous behaviour.

Signed-off-by: Dave Stevenson <[email protected]>
For downstream only, add back in the legacy single planar
SAND formats.

Signed-off-by: Dave Stevenson <[email protected]>
Upstream will take the multi-planar SAND format, but add back
in the downstream single planar variant for backwards compatibility

Signed-off-by: Dave Stevenson <[email protected]>
To avoid user complaints that /dev/video0 isn't their USB
webcam, add downstream patch that allows setting the preferred
video device number.

Signed-off-by: Dave Stevenson <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.