Skip to content

Stop max-fails crashes and adjust integration tests

Deborah Brouwer requested to merge dbrouwer/deqp-runner:max_fails into main

The max-fails option relies on the test group to crash to stop execution.
But if there are many test groups, this will cause many crashes and won't stop as expected.
See for example:
https://gitlab.freedesktop.org/mesa/mesa/-/jobs/63424434
https://gitlab.freedesktop.org/mesa/mesa/-/jobs/63424436
https://gitlab.freedesktop.org/mesa/mesa/-/jobs/63424337

Check if the maximum failures has been reached before processing each test group, so that each test group knows to stop without crashing.
Also stop counting unexpected improvements toward the max failures and stop looking for partial subtest names in list of expected failures. Also adjust the integration tests so that they would catch more of these kinds of failure.

Fixes issue #58 (closed)

/cc @daniels @gallo

Edited by Deborah Brouwer

Merge request reports