Skip to content

WIP: v4l2: Fix the transform_caps filtering

Mauro Carvalho Chehab requested to merge mchehab_kernel/gst-plugins-good:add_bayer+debug into master

The goal here is to discuss mchehab_kernel/gst-plugins-good@89fd92a6.

It solves a bug where V4L2 plugin returns an EMPTY caps when negotiating the capabilities of the following pipeline:

videotestsrc ! video/x-raw,format=RGB ! v4l2convert ! video/x-bayer,format=bggr ! bayer2rgb ! videoconvert ! xvimagesink

From my tests, and looking at the description at https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-libs/html/GstBaseTransform.html#GstBaseTransformClass for GstBaseTransformClass::transform_caps(), the current check at gst_v4l2_transform_transform_caps() is wrong, as it is using the caps from the wrong direction:

transform_caps ()  Optional. Given the pad in this direction and the given caps, what
                   caps are allowed on the other pad in this element ?

As a reference, I'm testing here with the vim2m patch I submitted to V4L2 ML: https://git.linuxtv.org/mchehab/experimental.git/commit/?h=vim2m&id=7fd6ccf110b7c167a2304ffd482e6c04252c4909.

With that, vim2m driver supports the following formats:

	Type: Video Output            # e. g. M2M source
	[0]: 'RGBP' (16-bit RGB 5-6-5)
	[1]: 'RGBR' (16-bit RGB 5-6-5 BE)
	[2]: 'RGB3' (24-bit RGB 8-8-8)
	[3]: 'BGR3' (24-bit BGR 8-8-8)
	[4]: 'YUYV' (YUYV 4:2:2)

	Type: Video Capture          # e. g. M2M sink
	[0]: 'RGBP' (16-bit RGB 5-6-5)
	[1]: 'RGBR' (16-bit RGB 5-6-5 BE)
	[2]: 'RGB3' (24-bit RGB 8-8-8)
	[3]: 'BGR3' (24-bit BGR 8-8-8)
	[4]: 'YUYV' (YUYV 4:2:2)
	[5]: 'BA81' (8-bit Bayer BGBG/GRGR)
	[6]: 'GBRG' (8-bit Bayer GBGB/RGRG)
	[7]: 'GRBG' (8-bit Bayer GRGR/BGBG)
	[8]: 'RGGB' (8-bit Bayer RGRG/GBGB)

From V4L2 driver's perspective, the above means that any "Video Output" source PAD format could be converted into any "Video Capture" input pad.

In other words, any query about what formats the M2M supports should consider valid:

  • for sink:
    • RGB 565 (LE/BE), RGB 24, BGR 24 and YUYV.
  • for source:
    • RGB 565 (LE/BE), RGB 24, BGR 24 and YUYV, plus Bayer ones (BGGR, GBRG, BRBG and RGGB).

As described at the GstBaseTransformClass::transform_caps(), when ->transform_caps() is called with the direction GST_PAD_SINK, it should actually return the valid source formats (and vice versa).

In other words, a call with: gst_v4l2_transform_transform_caps(btrans, GST_PAD_SINK, caps, filter) should get the GST_PAD_SOURCE caps, returing it, intercepted with filter. After the patch, the logs from ->transform_caps seem to be doing what it is expected to do:

gst_v4l2_transform_transform_caps: --->      direction: Sink
gst_v4l2_transform_transform_caps:   intersect from caps video/x-raw, format=(string)RGB, width=(int)320, height=(int)240, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)progressive, colorimetry=(string)1:1:5:1, pixel-aspect-ratio=(fraction)1/1
gst_v4l2_transform_transform_caps: intersect with filter (NULL)
gst_v4l2_transform_transform_caps:    intersect from pad video/x-bayer, format=(string){ bggr, gbrg, grbg, rggb }, width=(int)[ 1, 32768 ], height=(int)[ 1, 32768 ], framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw, format=(string){ RGB16, RGB, BGR, YUY2 }, width=(int)[ 1, 32768 ], height=(int)[ 1, 32768 ], framerate=(fraction)[ 0/1, 2147483647/1 ]
gst_v4l2_transform_transform_caps:     cleaned intersect video/x-bayer, framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw, framerate=(fraction)[ 0/1, 2147483647/1 ]
gst_v4l2_transform_transform_caps:           transformed video/x-raw, format=(string)RGB, width=(int)320, height=(int)240, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)progressive, colorimetry=(string)1:1:5:1, pixel-aspect-ratio=(fraction)1/1
gst_v4l2_transform_transform_caps:                  into video/x-bayer, framerate=(fraction)[ 0/1, 2147483647/1 ]; video/x-raw, framerate=(fraction)[ 0/1, 2147483647/1 ]

At least on my eyes, it sounds that it is doing the right thing after the patch.

Merge request reports