XTS test XCopyColormapAndFree/5 hangs with commit 99a2cf1a

Some tests in the X test suite (xts) which used to pass are now hanging while running.

I managed to identify one test in particular that would reproducibly hang:

$ TET_RESFILE=/tmp/XCopyColormapAndFree.log /usr/libexec/xts5/Xlib7/XCopyColormapAndFree 5

Then checked in gdb to find out where:

(gdb) bt
#0  futex_wait (futex_word=0x556f9be7b890, expected=2, private=0) at ../sysdeps/nptl/futex-internal.h:146
#1  __GI___lll_lock_wait (futex=futex@entry=0x556f9be7b890, private=0) at lowlevellock.c:49
#2  0x00007f610cfb9791 in lll_mutex_lock_optimized (mutex=0x556f9be7b890) at pthread_mutex_lock.c:48
#3  ___pthread_mutex_lock (mutex=0x556f9be7b890) at pthread_mutex_lock.c:93
#4  0x00007f610d151b77 in _XLockDisplay (dpy=0x556f9be7fa50) at /usr/src/debug/libX11-1.8.9-1.fc40.x86_64/src/locking.c:470
#5  0x00007f610d16c290 in XGetVisualInfo (dpy=0x556f9be7fa50, visual_info_mask=2, visual_info_template=0x7ffdeab255c0, nitems=0x7ffdeab25554)
    at /usr/src/debug/libX11-1.8.9-1.fc40.x86_64/src/VisUtil.c:73
#6  0x00007f610d1b6b65 in CmapRecForColormap (dpy=dpy@entry=0x556f9be7fa50, cmap=<optimized out>) at xcms/cmsCmap.c:143
#7  0x00007f610d1b6e5c in _XcmsCopyCmapRecAndFree (dpy=0x556f9be7fa50, src_cmap=<optimized out>, copy_cmap=41943290) at xcms/cmsCmap.c:299
#8  0x00007f610d14be7b in XCopyColormapAndFree (dpy=0x556f9be7fa50, src_cmap=41943289) at /usr/src/debug/libX11-1.8.9-1.fc40.x86_64/src/CopyCmap.c:62
#9  0x0000556f9bbd2f75 in t005 () at /usr/src/debug/xts-20240529-1.fc40.x86_64/xts5/Xlib7/XCopyColormapAndFree.c:771
#10 0x0000556f9bbd3786 in tet_invoketp (icnum=5, tpnum=<optimized out>) at ../../src/tet3/apilib/tciface.c:218
#11 call_1tp (icnum=icnum@entry=5, tpnum=tpnum@entry=1, testnum=5) at ../../src/tet3/tcm/dtcm.c:526
#12 0x0000556f9bbd1144 in call_tps (icnum=5, tpcount=<synthetic pointer>) at ../../src/tet3/tcm/dtcm.c:425
#13 tet_tcm_main (argc=<optimized out>, argv=<optimized out>) at ../../src/tet3/tcm/dtcm.c:375
#14 main (argc=<optimized out>, argv=<optimized out>) at ../../src/tet3/tcm/tcm.c:152

So it's hanging on the futex used from the _XLockDisplay() / _XUnlockDisplay() calls.

So my first reaction to that was to build xts with XTEST_THREADS defined to have xts call XInitThreads() itself, but that did not help at all with the hang.

Breaking on _XLockDisplay() and _XUnlockDisplay() in gdb didn't prove to be practical, those get called hundreds of times.

So I tried breaking on XInitThreads() and realized that it was called from the constructor (commit afcdb6fb) so setting XTEST_THREADS in xts would make no difference.

So I tried building libX11 with --disable-thread-safety-constructor and bingo!, no more hangs!

To be perfectly honest, I do not know if that particular xts test is doing something wrong, this is all generated code without any documentation (that I could find) on how that works in xts, but looking at the generated code in xts for that particular test doesn't reveal any obvious issue with the code itself.

Anyhow, this is something that worked before and fails now, so that qualifies as a regression, I guess. I can imagine the same would affect real applications as well.