xfree86/modes: Use correct crtc gamma size in xf86RandR12InitGamma
When first initializing the DIX layer's RRCrtc structures, xf86RandR12Init12 hard-codes a gamma ramp size of 256 elements and then calls into the DDX layer to copy the gamma ramps from the RRCrtc into the xf86Crtc's gamma ramp arrays. When it does this, it assumes that crtc->gamma_size == crtc->randr_crtc->gammaSize == 256. However, commit 245b9db0 modified the modesetting driver to update crtc->gamma_size with the true size of the hardware's gamma ramp when the DRM GAMMA_LUT property is available. This causes xf86RandR12CrtcSetGamma to read past the end of the randr_crtc->gamma{Red,Green,Blue} arrays:
PreInit
drmmode_pre_init
drmmode_crtc_init
crtc->gamma_size = 1024
ScreenInit
xf86CrtcScreenInit
xf86RandR12Init
xf86RandR12Init12
xf86RandR12CreateObjects12
RRCrtcCreate
randr_crtc->gammaSize = 0
xf86RandR12InitGamma(pScrn, 256)
RRCrtcGammaSetSize
randr_crtc->gammaSize = 256
xf86RandR12InitGamma
xf86RandR12CrtcInitGamma
RRCrtcGammaSet
xf86RandR12CrtcSetGamma
// crtc->gamma_size is 1024 here, while randr_crtc->gammaRed
// is a 256-element array.
memcpy(crtc->gamma_red, randr_crtc->gammaRed, crtc->gamma_size * sizeof(crtc->gamma_red[0]));
drmmode_setup_colormap
xf86HandleColormaps
xf86RandR12InitGamma
RRCrtcGammaSetSize
randr_crtc->gammaSize = 1024
Rather than passing gammaSize as an argument to xf86RandR12InitGamma, just use the xf86Crtc's gamma size as the argument to RRCrtcGammaSetSize so that the RRCrtc and xf86Crtc agree about the size of the gamma ramp immediately.
Fixes: 245b9db0 - modesetting: Use GAMMA_LUT when available
Closes: #1126 (closed)
Signed-off-by: Aaron Plattner aplattner@nvidia.com