Skip to content

modesetting: Defer crtc gamma size upgrade to drmmode_setup_colormap

Aaron Plattner requested to merge aplattner/xserver:defer-gamma-upgrade into master

Rather than trying to create a gamma ramp array of the appropriate size in drmmode_crtc_init when the GAMMA_LUT property should be used, just flag the crtc as wanting to use the GAMMA_LUT property and then replace the gamma ramp later, right before calling xf86HandleColormaps. This avoids a problem during initial startup where xf86RandR12CreateObjects12 hard-codes a gamma ramp size of 256, causing xf86RandR12CrtcSetGamma to read past the end of the DIX layer's RandR gamma ramp array:

  PreInit
    drmmode_pre_init
      drmmode_crtc_init
        crtc->gamma_size = 1024

  ScreenInit
    xf86CrtcScreenInit
      xf86RandR12Init
        xf86RandR12Init12
          xf86RandR12CreateObjects12
            RRCrtcCreate
              randr_crtc->gammaSize = 0
          xf86RandR12InitGamma(pScrn, 256)
            RRCrtcGammaSetSize
              randr_crtc->gammaSize = 256
          xf86RandR12InitGamma
            xf86RandR12CrtcInitGamma
              RRCrtcGammaSet
                xf86RandR12CrtcSetGamma
                  // crtc->gamma_size is 1024 here, while randr_crtc->gammaRed
                  // is a 256-element array.
                  memcpy(crtc->gamma_red, randr_crtc->gammaRed, crtc->gamma_size * sizeof(crtc->gamma_red[0]));
    drmmode_setup_colormap
      xf86HandleColormaps
        xf86RandR12InitGamma
          RRCrtcGammaSetSize
            randr_crtc->gammaSize = 1024

Fixes: 245b9db0 - modesetting: Use GAMMA_LUT when available Closes: #1126 (closed) Signed-off-by: Aaron Plattner aplattner@nvidia.com

Merge request reports