Skip to content

Draft: Color Pipeline API w/ VKMS

Harry Wentland requested to merge color-pipeline-wip-1 into color-pipeline-wip-base

This is an RFC set for a color pipeline API, along with a sample implementation in VKMS. All the key API bits are here. VKMS now supports two named transfer function colorops and two matrix colorops. We have IGT tests that check all four of these colorops with a pixel-by-pixel comparison that checks that these colorops do what we expect them to do with a +/- 1 8 bpc code point margin.

The big new change with v4 is the addition of an amdgpu color pipeline, for all AMD GPUs with DCN 3 and newer. Amdgpu now support the following:

  1. 1D Curve EOTF
  2. 3x4 CTM
  3. Multiplier
  4. 1D Curve Inverse EOTF
  5. 1D LUT
  6. 1D Curve EOTF
  7. 1D LUT

The supported curves for the 1D Curve type are:

  • sRGB EOTF and its inverse
  • PQ EOTF, scaled to [0.0, 125.0] and its inverse
  • BT.2020/BT.709 OETF and its inverse

Note that the 1st and 5th colorops take the EOTF or Inverse OETF while the 3rd colorop takes the Inverse EOTF or OETF.

We are working on two more ops for amdgpu, the HDR multiplier and the 3DLUT, which will give us this:

  1. 1D Curve EOTF
  2. 3x4 CTM
  3. HDR Multiplier
  4. 1D Curve Inverse EOTF
  5. 1D LUT
  6. 3D LUT
  7. 1D Curve EOTF
  8. 1D LUT

This, essentially mirrors the color pipeline used by gamescope and presented by Melissa Wen, with the exception of the DEGAM LUT, which is not currently used. See [1] https://indico.freedesktop.org/event/4/contributions/186/attachments/138/218/xdc2023-TheRainbowTreasureMap-MelissaWen.pdf

After this we'd like to also add the following ops:

  • Scaler (Informational only)
  • Color Encoding, to replace drm_plane's COLOR_ENCODING
  • Color Range, to replace drm_plane's COLOR_RANGE

This patchset is grouped as follows:

  • Patches 1-3: couple general patches/fixes
  • Patches 4-7: introduce kunit to VKMS
  • Patch 7: description of motivation and details behind the Color Pipeline API. If you're reading nothing else but are interested in the topic I highly recommend you take a look at this.
  • Patches 7-27: DRM core and VKMS changes for color pipeline API
  • Patches 28-40: DRM core and amdgpu changes for color pipeline API

VKMS patches could still be improved in a few ways, though the payoff might be limited and I would rather focus on other work at the moment. The most obvious thing to improve would be to eliminate the hard-coded LUTs for identity, and sRGB, and replace them with fixed-point math instead.

There are plenty of things that I would like to see here but haven't had a chance to look at. These will (hopefully) be addressed in future iterations, either in VKMS or amdgpu:

  • Clear documentation for each drm_colorop_type
  • Add custom LUT colorops to VKMS
  • Add pre-blending 3DLUT
  • How to support HW which can't bypass entire pipeline?
  • Add ability to create colorops that don't have BYPASS
  • Can we do a LOAD / COMMIT model for LUTs (and other properties)?
  • read-only scaling colorop which defines scaling taps and position
  • read-only color format colorop to define supported color formats for a pipeline
  • named matrices, for things like converting YUV to RGB

IGT tests can be found at igt-gpu-tools!1

IGT patches are also being sent to the igt-dev mailing list.

If you prefer a gitlab MR for review you can find it at !5

v4:

  • Add amdgpu color pipeline (WIP)
  • Don't block setting of deprecated properties, instead pass client cap to atomic check so drivers can ignore these props
  • Drop IOCTL definitions (Pekka)
  • Use enum property for colorop TYPE (Pekka)
  • A few cleanups to the docs (Pekka)
  • Rework the TYPE enum to name relation to avoid code duplication (Pekka)
  • Add missing function declarations (Chaitanya Kumar Borah)
  • Allow setting of NEXT property to NULL in set function (Chaitanya Kumar Borah)
  • Add helper for creation of pipeline drm_plane property (Pekka)
  • Always create Bypass pipeline (Pekka)
  • A bunch of changes to VKMS kunit tests (Pekka)
  • Fix index in CTM doc (Pekka)

v3:

  • Abandon IOCTLs and discover colorops as clients iterate the pipeline
  • Remove need for libdrm
  • Add color_pipeline client cap and make mutually exclusive with COLOR_RANGE and COLOR_ENCODING properties
  • add CTM colorop to VKMS
  • Use include way for kunit testing static functions (Arthur)
  • Make TYPE a range property
  • Move enum drm_colorop_type to uapi header
  • and a bunch of smaller bits that are highlighted in the relevant commit description

v2:

  • Rebased on drm-misc-next
  • Introduce a VKMS Kunit so we can test LUT functionality in vkms_composer
  • Incorporate feedback in color_pipeline.rst doc
  • Add support for sRGB inverse EOTF
  • Add 2nd enumerated TF colorop to VKMS
  • Fix LUTs and some issues with applying LUTs in VKMS
Edited by Harry Wentland

Merge request reports