CM&HDR implementation roadmap

This is a rough plan on how the Weston implementation for color management and HDR support will be upstreamed. The upstreaming is split into two major phases:

• Features that can be implemented without any protocol extensions by assuming that client content is always standard sRGB.
• Implementing CM&HDR protocol extension and follow-up work.

There is an earlier plan at #348 (closed).

The CM&HDR protocol development is tracked at wayland-protocols!14.

This plan is likely incomplete and I try to keep it updated.

Features without explicit CM&HDR protocol:

• Output damage test. (!529 (merged))
• Prepare GL-renderer to be able to deal with dozens of GLSL shader variants and introduce a floating-point shadow framebuffer. (!132 (merged))
• Polish existing GLSL code and implement shader garbage collection. (#462 (closed), !552 (merged), !565 (merged))
• More GL-renderer refactoring: gl_shader_config. (!575 (merged))
• libweston internal data structure weston_paint_node. (!612 (merged))
• Framework for color management modules, with a no-op module for no color management. (!582 (merged), !592 (merged))
• Gamma-correct blending. (!596 (merged))
• Framework for color profile objects. (!637 (merged))
  • Non-ICC color profile API. (!1464 (merged))
• Color management module using LittleCMS, color transformation support in GL-renderer, using sRGB input and output profiles.
  • 3D LUT for the general case. (!723 (merged), !640 (merged))
  • Color pipeline optimizer. (!906 (merged), !1271 (merged), !1469 (merged))
  • Special cases:
    • 3x3 matrix + offset vector color space conversion (!906 (merged))
      • Ensure 3D LUT case keeps being tested (!820 (closed), !872 (merged))
      • Add unit tests for optimize_float_pipeline() (!1271 (merged))
    • type 4 transfer function (!1390 (merged))
    • Perceptual Quantizer transfer function and its inverse
    • Parameterized Hybrid Log-Gamma transfer function and its inverse
  • Algorithm for discovering linearization transfer function from non-decomposable ICC profile. (!640 (merged))
    • Tests for the discovery algorithm with blending. (!820 (closed), !864 (closed), !872 (merged), !925 (merged))
  • Add a log scope for color transformations: list all cached transformations, explain optimization in progress (!1186 (merged), !1245 (merged), !1297 (merged))
  • Verified correctness of ICC profile usage and rendering intents using https://displaycal.net/icc-color-management-test/ .
• Stop KMS planes from being used with client buffers when color management is on. (!582 (merged))
• Ability to set per-output ICC profiles and use them to convert sRGB content to monitor-specific colorspace. Not just VCGT but using the actual monitor color profile. (!637 (merged), !640 (merged))
  • Test VCGT handling via GL-renderer. (!1226 (merged))
• Share 3x3 matrix code between core, color-lcms, and tests.
• Tests for input-to-output category of color transformations (output decorations). (!1020 (merged), !1025 (merged))
• Compositor fallback rendering intent. (!1491 (merged))
• Option to override Xwayland window image description and rendering intent to allow using X11 color managed and profiling applications.
• Monitor colorimetry (non-ICC):
  • Get colorimetry description from weston.ini. (!856 (merged))
  • Get colorimetry description from EDID. (!583 (closed))
  • Convert colorimetry description into struct cmlcms_color_profile
• Experimental option to set a monitor into HDR mode in DRM-backend.
  • EOTF mode option (!835 (merged))
  • KMS HDR metadata, produced by color manager. (!856 (merged))
  • Use high bits-per-channel framebuffer and adjust max bpc KMS connector property automatically for HDR. (e6a9e3c4, #612 (closed), !900 (merged))
  • Program CRTC properties DEGAMMA, CTM and GAMMA to identity when not off-loading a color transformation to KMS. (!952 (merged))
  • Program Colorspace KMS property. (!1505 (merged))
• HDR brightness and black level controls.
• Optimize output color parameters for surface - API. Backend/core can ask color manager to provide the optimal output configuration (EOTF mode, HDR metadata, color transformation) to best present a certain surface. To be triggered on exclusive fullscreen surfaces.
• Handle untagged surfaces
  • Make the difference between untagged and sRGB tagged. (!1370 (merged))
  • Use a parametric profile for stock sRGB.

Features with CM&HDR protocol extension:

• Common protocol interfaces and ICC files. (!1356 (merged), !1501 (merged))
• Protocol interfaces: parametric image descriptions. (!1502)
• Protocol test client for ICC profiles embedded in images. (!1499 (merged))
• Protocol test client for parametric image descriptions.
• Colorimetric transformations, SDR and HDR:
  • parametric source, parametric destination
  • parametric source, ICC destination
  • ICC source, parametric destination
  • colorimetric rendering intents
  • perceptual rendering intent, entry level / simple
  • saturation rendering intent

Further features:

• Ensure turning CRTC on in KMS uses pass-through color pipeline (reference, !952 (merged)).
• Offload post-blending EOTF⁻¹ to KMS hardware.
  • Rewrite old-school gamma LUT handling to go through color manager and use the atomic KMS state tracking.
  • Supersede cms-static and cms-colord plugins, as the post-blending LUT in KMS is actually EOTF⁻¹ + VCGT.
• Allow using KMS planes with client buffers when KMS features are sufficient.
• Support for monitor profile measurement (needs protocol).

Color calibration auditing system

A framework that allows collecting everything that might affect color reproduction:

• Monitor identity.
• Monitor connection type.
• Monitor settings, e.g. via DDC/CI.
• All recognized DRM KMS properties with their values.
• All unrecognized DRM KMS properties with their values. "Unrecognized" means Weston has no explicit code to control that property. A change in any property value could imply a change in color reproduction. Also missing or new properties could imply a change in color reproduction.
• Renderer identity: GL ES implementation and the implementation version, GPU model, GPU driver version...
• Weston/libweston version.
• Custom environmental sensors, e.g. lighting?

If any change in the above is detected, the user is warned that the monitor profile they measured and are using may not be currently valid.