Supported formats not considered during PreInit
From #xorg-devel:
14:24 < shadeslayer> hey, is there a config option to set the pixel format in Xorg to BGR8888?
14:30 < ajax> shadeslayer: i don't think so? i haven't checked, but i know most of the drivers set
channel masks unconditionally
14:36 < shadeslayer> ajax: ah damn
14:38 < ajax> shadeslayer: what do you need it for?
14:38 < shadeslayer> ajax: one of the ARM boards I'm using only accepts BG24 on the DRM side
14:38 < shadeslayer> in the drm debug log we see : [ 8.907086] [drm:drm_atomic_check_only] Invalid
pixel format RG24 little-endian (0x34324752)
14:39 < shadeslayer> and on the modetest side of things, passing 30:480x800@BG24 works
14:39 < ajax> is there a way we could discover that programmatically?
14:40 < shadeslayer> I have a dts file here
14:40 < ajax> how about, like, an ioctl on the drm device
14:40 < shadeslayer> ajax: xlnx,vformat = "BG24"; < that's what defined
14:41 < shadeslayer> ajax: maybe, not sure how to do that
14:41 * ajax digs a bit
14:45 < ajax> shadeslayer: does this happen to be mali?
14:45 < shadeslayer> ajax: yes
14:56 < ajax> shadeslayer: so. we do already have code to get the supported formats out of the
kernel, but we're not using that knowledge in the modesetting driver when we set up
the channel masks
14:56 < shadeslayer> ajax: ah, I'm using the armsoc driver here
14:57 < ajax> well, okay, probably the same bug there
14:57 < shadeslayer> ajax: can I somehow make it set a prefered format?
14:58 < ajax> https://github.com/endlessm/xf86-video-armsoc/blob/master/src/armsoc_driver.c#L1009
14:58 < ajax> ^ that should be a call instead to miSetVisualTypesAndMasks()
14:59 < ajax> which takes three additional arguments, the r/g/b masks. can either hardcode what
you need or compute it from the supported formats from the drm device
14:59 < shadeslayer> aha cool
15:01 < shadeslayer> ajax: what would be a hardcoded value for BG24?
15:03 < ajax> r 0xff, g 0xff00, b 0xff0000
15:03 < shadeslayer> thanks!!
15:04 < ajax> or the reverse, or that all << 8.
15:04 < ajax> kiiiinda guessing here. there's not many drivers that use that call so i don't have
examples to cheat from
15:04 < shadeslayer> fair enough
15:05 < ajax> can always set a breakpoint in gdb on that call (which miSetVisualTypes is
implemented in terms of) and see what masks it computes
15:06 < ajax> no promises about clients handling it sanely, either
Edited by Adam Jackson