Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
media: rp1: Add downstream CFE (Camera Front End) driver
Signed-off-by: Naushir Patuck <[email protected]> v4l2: Add pisp compression format support to v4l2 Signed-off-by: Naushir Patuck <[email protected]> media: rp1: cfe: Fix use of freed memory on errors cfe_probe_complete() calls cfe_put() on both success and fail code paths. This works for the success path, but causes the cfe_device struct to be freed, even if it will be used later in the teardown code. Fix this by making the ref handling a bit saner: Let the video nodes have the refs as they do now, but also keep a ref in the "main" driver, released only at cfe_remove() time. This way the driver does not depend on the video nodes keeping the refs. Signed-off-by: Tomi Valkeinen <[email protected]> media: rp1: cfe: Fix width & height in cfe_start_channel() The logic for handling width & height in cfe_start_channel() is somewhat odd and, afaics, broken. The code reads: bool start_fe = is_fe_enabled(cfe) && test_all_nodes(cfe, NODE_ENABLED, NODE_STREAMING); if (start_fe || is_image_output_node(node)) { width = node->fmt.fmt.pix.width; height = node->fmt.fmt.pix.height; } cfe_start_channel() is called for all video nodes that will be used. So this means that if, say, fe_stats is enabled as the last node, start_fe will be true, and width and height will be taken from fe_stats' node. The width and height will thus contain garbage, which then gets programmed to the csi2 registers. It seems that this often still works fine, though, probably if the width & height are large enough. Drop the above code, and instead get the width & height from the csi2 subdev's sink pad for the csi2 channel that is used. For metadata the width & height will be 0 as before. Signed-off-by: Tomi Valkeinen <[email protected]> media: rp1: cfe: Fix verbose debug print The debug print in cfe_schedule_next_csi2_job() is printed every frame, and should thus use cfe_dbg_irq() to avoid spamming, rather than cfe_dbg(). Signed-off-by: Tomi Valkeinen <[email protected]> media: rp1: cfe: Rename xxx_dbg_irq() to xxx_dbg_verbose() Rename the xxx_dbg_irq() macros to xxx_dbg_verbose(), as they can be used to verbose debugs outside irq context too. Signed-off-by: Tomi Valkeinen <[email protected]> media: rp1: cfe: Add verbose debug module parameter Expose the verbose debug flag as a module parameter. Signed-off-by: Tomi Valkeinen <[email protected]> media: rp1: cfe: Drop unused field Drop 'sensor_embedded_data' field, as it is unused. Signed-off-by: Tomi Valkeinen <[email protected]> media: rp1: cfe: Fix default meta format's field Set default meta format's field to V4L2_FIELD_NONE, instead of zeroing it which indicates V4L2_FIELD_ANY. Metadata doesn't have fields, so NONE makes sense, and furthermore the default v4l2 link validation will check for matching fields, or that the sink field is NONE. Signed-off-by: Tomi Valkeinen <[email protected]> media: rp1: cfe: Fail streaming if FE_CONFIG node is not enabled When the FE is enabled, ensure that the FE_CONFIG node is enabled. Otherwise fail cfe_start_streaming() entirely. Signed-off-by: Naushir Patuck <[email protected]> drivers: media: rp1_cfe: Remove PISP specific MBUS formats Remove the MEDIA_BUS_FMT_PISP* format codcs entirely. For the image pad formats, use the 16-bit Bayer format mbus codes instead. For the config and stats pad formats, use MEDIA_BUS_FMT_FIXED. Signed-off-by: Naushir Patuck <[email protected]> drivers: media: rp1_cfe: Fix link validate test for pixel format Now that we have removed unique PISP media bus codes, the cfe format table has multiple entries with the same media bus code for 16-bit formats. The test in cfe_video_link_validate() did not account for this. Fix it by testing the media bus code and the V4L2 pixelformat 4cc together. As a drive-by, ensure we have a valid CSI2 datatype id when programming the hardware block. Signed-off-by: Naushir Patuck <[email protected]> drivers: media: cfe: Set the CSI-2 link frequency correctly Use the sensor provided link frequency to set the DPHY timing parameters on stream_on. This replaces the hard-coded 999 MHz value currently being used. As a fallback, revert to the original 999 Mhz link frequency. As a drive-by, fix a 80-character line formatting error. Signed-off-by: Naushir Patuck <[email protected]> drivers: media: cfe: Don't confuse MHz and Mbps The driver was interchaning these units when talking about link rate. Fix this to avoid confusion. Apart from the logging message change, there is no function change in this commit. Signed-off-by: Naushir Patuck <[email protected]> media: rp1: csi2: Fix missing reg writes The driver has two places where it writes a register based on a condition, and when that condition is false, the driver presumes that the register has the reset value. This is not a good idea, so fix those places to always write the register. Signed-off-by: Tomi Valkeinen <[email protected]> media: rp1: fe: Use ~0, not -1, when working with unsigned values Use ~0, not -1, when working with unsigned values (-1 is not unsigned). Signed-off-by: Tomi Valkeinen <[email protected]> media: rp1: Add back reg write debug prints Add back debug prints in csi2 and pisp_fe reg_write() functions, but use the 'irq' variants to avoid spamming in normal situation. Signed-off-by: Tomi Valkeinen <[email protected]> media: rp1: csi2: Track CSI-2 errors Track the errors from the CSI-2 receiver: overflows and discards. These are recorded in a table which can be read by the userspace via debugfs. As tracking the errors may cause much more interrupt load, the tracking needs to be enabled with a module parameter. Note that the recording is not perfect: we only record the last discarded DT for each discard type, instead of recording all of them. This means that e.g. if the device is discarding two unmatched DTs, the debugfs file only shows the last one recorded. Recording all of them would need a more sophisticated recording system to avoid the need of a very large table, or dynamic allocation. Signed-off-by: Tomi Valkeinen <[email protected]> media: rp1: csi2: Set values for enum csi2_mode Set hardcoded values for enum csi2_mode, as the values will be programmed to HW registers. Signed-off-by: Tomi Valkeinen <[email protected]> media: rp1: fe: Fix default mbus code When pisp_fe_pad_set_fmt() is given an mbus code that CFE does not support, it currently defaults to MEDIA_BUS_FMT_SBGGR10_1X10. This is not correct, as FE does not support SBGGR10. Set the default to MEDIA_BUS_FMT_SRGGB16_1X16 instead. Signed-off-by: Tomi Valkeinen <[email protected]> drivers: media: cfe: Find the source pads on the sensor entity The driver was assuming that pad 0 on the sensor entity was the appropriate source pad, but this isn't necessarily the case. With video-mux, it has the sink pads first, and then the source pad as the last one. Iterate through the sensor pads to find the relevant source pads. Signed-off-by: Dave Stevenson <[email protected]> media: rp1: cfe: Expose find_format_by_pix() Make find_format_by_pix() accessible to other files in the driver. Signed-off-by: Tomi Valkeinen <[email protected]> media: rp1: cfe: Add cfe_find_16bit_code() and cfe_find_compressed_code() Add helper functions which, given an mbus code, return the 16-bit remapped mbus code or the compressed mbus code. Signed-off-by: Tomi Valkeinen <[email protected]> media: rp1: csi2: Use get_frame_desc to get CSI-2 VC and DT Use get_frame_desc pad op for asking the CSI-2 VC and DT from the source device driver, instead of hardcoding to VC 0, and getting the DT from a formats table. To keep backward compatibility with sources that do not implement get_frame_desc, implement a fallback mechanism that always uses VC 0, and gets the DT from the formats table, based on the CSI2's sink pad's format. Signed-off-by: Tomi Valkeinen <[email protected]> media: rp1: cfe: Add is_image_node() The hardware supports streaming from memory (in addition to streaming from the CSI-2 RX), but the driver does not support this at the moment. There are multiple places in the driver which uses is_image_output_node(), even if the "output" part is not relevant. Thus, in a minor preparation for the possible support for streaming from memory, and to make it more obvious that the pieces of code are not about the "output", add is_image_node() which will return true for both input and output video nodes. While at it, reformat also the metadata related macros to fit inside 80 columns. Signed-off-by: Tomi Valkeinen <[email protected]> media: rp1: cfe: Dual purpose video nodes The RP1 CSI-2 DMA can capture both video and metadata just fine, but at the moment the video nodes are only set to support either video or metadata. Make the changes to support both video and metadata. This mostly means tracking both video format and metadata format separately for each video node, and using vb2_queue_change_type() to change the vb2 queue type when needed. Briefly, this means that the user can get/set both video and meta formats to a single video node. The vb2 queue buffer type will be changed when the user calls REQBUFS or CREATE_BUFS ioctls. This buffer type will be then used as the "mode" for the video node when the user starts the streaming, and based on that either the video or the meta format will be used. A bunch of macros are added (node_supports_xxx()), which tell if a node can support a particular mode, whereas the existing macros (is_xxx_node()) will tell if the node is currently in a particular mode. Note that the latter will only work correctly between the start of the streaming and the end of the streaming, and thus should be only used in those code paths. However, as the userspace (libcamera) does not support dual purpose video nodes, for the time being let's keep the second video node as V4L2_CAP_META_CAPTURE only to keep the userspace working. Signed-off-by: Tomi Valkeinen <[email protected]> media: rp1: Drop LE handling The driver registers for line-end interrupts, but never uses them. This just causes extra interrupt load, with more complexity in the driver. Drop the LE handling. It can easily be added back if later needed. Signed-off-by: Tomi Valkeinen <[email protected]> media: rp1: cfe: Improve link validation for metadata Improve the link validation for metadata by: - Allowing capture buffers that are larger than the incoming frame (instead of requiring exact match). - Instead of assuming that a metadata unit ("pixel") is 8 bits, use find_format_by_code() to get the format and use the bit depth from there. E.g. bit depth for RAW10 metadata will be 10 bits, when we move to the upstream metadata formats. Signed-off-by: Tomi Valkeinen <[email protected]> drivers: media: cfe: Add more robust ISR handlers Update the ISR logic to be more robust to sensors in problematic states where interrupts may start arriving overlapped and/or missing. 1) Test for cur_frame in the FE handler, and if present, dequeue it in an error state so that it does not get orphaned. 2) Move the sequence counter and timestamp variables to the node structures. This allows the ISR to track channels running ahead when interrupts arrive unordered. 3) Add a test to ensure we don't have a spurios (but harmlesS) call to the FE handler in some circumstances. Signed-off-by: Naushir Patuck <[email protected]> media: rp1: cfe: Fix error paths in cfe_start_streaming Noted that if we get "node link is not enabled", then we also get the videobuf2 splat for the driver not cleaning up correctly on a failed start_streaming, and indeed we weren't returning the buffers. Checking the other error paths, noted that the "FE enabled, but FE_CONFIG node is not" path was not calling pm_runtime_put. Fix both paths. Signed-off-by: Dave Stevenson <[email protected]> drivers: media: cfe: Increase default size of embedded buffer Increase the size of the default embedded buffer to 16k. This is done to match what is advertised by the IMX219 driver and workaround a problem where the embedded stream is not actually used. Without full streams API support, the media pipeline validation will fail in these circumstances. Signed-off-by: Naushir Patuck <[email protected]> media: rp1: cfe: Actually use the number of lanes configured The driver was calling get_mbus_config to ask the sensor subdev how many CSI2 data lanes it wished to use and with what other properties, but then failed to pass that to the DPHY configuration. Signed-off-by: Dave Stevenson <[email protected]> media: rp1: csi2: Fix csi2_pad_set_fmt() The CSI-2 subdev's set_fmt currently allows setting the source and sink pad formats quite freely. This is not right, as the CSI-2 block can only do one of the following when processing the stream: 1) pass through as is, 2) expand to 16-bits, 3) compress. The csi2_pad_set_fmt() should take this into account, and only allow changing the source side mbus code, compared to the sink side format. Signed-off-by: Tomi Valkeinen <[email protected]> media: rp1: csi2: Use standard link_validate The current csi2_link_validate() skips some important checks. Let's rather use the standard v4l2_subdev_link_validate_default() as the link_validate hook. Signed-off-by: Tomi Valkeinen <[email protected]> media: rp1: csi2: Squash fixes media: rp1: fe: Fix pisp_fe_pad_set_fmt() pisp_fe_pad_set_fmt() allows setting the pad formats quite freely. This is not correct, and the function should only allow formats as supported by the hardware. Fix this by: Allow no format changes for FE_CONFIG_PAD and FE_STATS_PAD. They should always be the hardcoded initial ones. Allow setting FE_STREAM_PAD freely (but the mbus code must be supported), and propagate the format to the FE_OUTPUT0_PAD and FE_OUTPUT1_PAD pads. Allow changing the mbus code for FE_OUTPUT0_PAD and FE_OUTPUT1_PAD pads only if the mbus code is the compressed version of the sink side code. TODO: FE supports scaling and cropping. This should be represented here too? Signed-off-by: Tomi Valkeinen <[email protected]> media: rp1: fe: Use standard link_validate The current pisp_fe_link_validate() skips some important checks. Let's rather use the standard v4l2_subdev_link_validate_default() as the link_validate hook. Signed-off-by: Tomi Valkeinen <[email protected]> drivers: media: cfe: Add 16-bit and compressed mono format support Signed-off-by: Naushir Patuck <[email protected]> media: rp1: cfe: Add missing remaps 8-bit bayer formats are missing remap definitions. Add them. Signed-off-by: Tomi Valkeinen <[email protected]> media: rp1: cfe: Add missing compressed remaps 16-bit bayer formats are missing compressed remap definitions. Add them. Signed-off-by: Tomi Valkeinen <[email protected]> drivers: media: pisp_be: pisp_fe: Update UAPI header licenses Update the license tags on the pisp UAPI header files with the "Linux-syscall-note" clause. Also replace the "GPL-2.0" tag with the preferred "GPL-2.0-only" tag. Signed-off-by: Naushir Patuck <[email protected]> media: rp1: cfe: Use the MIPI_CSI2_DT_xxx defines for csi_dt Seeing as we now have the CSI2 data types defined, make use of them instead of hardcoding the values. Signed-off-by: Dave Stevenson <[email protected]> media: rp1: cfe: Add a csi_dt value for 16bit formats Raw 16bit formats didn't have a csi_dt value defined, which presumably would trip the WARN_ON(!fmt->csi_dt); in cfe_start_channel. The value is defined in CSI2 v2.0 as 0x2e, so set it accordingly. Signed-off-by: Dave Stevenson <[email protected]> drivers: media: pisp_be: Add mono and 48-bit RGB pixel format support Signed-off-by: Naushir Patuck <[email protected]> drivers: media: pisp_be: Update seqeuence numbers of the buffers Add a framebuffer sequence counter and increment on every completed job. This counter is then used to update the VB2 buffer sequence count before calling vb2_buffer_done(). Signed-off-by: Naushir Patuck <[email protected]> drivers: media: cfe: Add remap entries for mono formats The 8-bit and 16-bit mono formats were missing the appropriate remap entries in the format table. Signed-off-by: Naushir Patuck <[email protected]> media: rp1-cfe: Fix up link validation for CFE CFG input After commit 5fd3e24 ("media: v4l2-subdev: Support hybrid links in v4l2_subdev_link_validate()") link_validate is called on V4L2 OUTPUT devices such as the CFE cfg buffers input. The CFE link_validate function was assuming it was always the sink of a link, which goes wrong on that port and does an invalid dereference. Signed-off-by: Dave Stevenson <[email protected]>
- Loading branch information
