Skip to content

nvk: Rework TLS and image/sampler table handling

Faith Ekstrand requested to merge gfxstrand/mesa:nvk/tls-table-rework into nouveau/vk

When Dave originally added TLS, he did so with a preamble pushbuf which sets up the TLS area prior to executing any actual command buffers. This has the advantage that the TLS BO is never directly referenced in any command buffer and we can swap it out even after command buffers have been built. When I added image/sampler tables, I sort-of went along with it but then did something different for 3D.

Thanks to buffer reference counting, we can employ a fairly simple locking scheme. Each of the globals: images, samplers, and TSL has a lock which gets taken whenever the table is modified. When we go to do a submit, we take each lock and take a queue-local reference to the BO and grab a copy of its size parameter under the lock. If anything has changed, we update the queue-local preamble pushbuf. Because the queue also holds a reference, we can safely proceed to submit command buffers which reference those global BOs without the individual global locks taken. If TLS or one of the descriptor tables resizes while we're mid-submit, our reference to the BO will remain valid until either the destruction of the queue or the next submit.

The one small bit of threaded cleverness (one always has to be careful with that) is that the TLS size can be checked without taking the lock because the TLS size only ever increases.

NOTE:

This is marked draft because it currently blows up CTS runs. In particular, it DOS's the GPU and nouveau can't recover. I think this isn't a real regression and is more that tests are suddenly running that couldn't get as far before and it's uncovering hidden bugs. However, I have yet to prove that. It may be buggy in and of itself.

Edited by Faith Ekstrand

Merge request reports