Add test-suite support for deqp-runner.
Lets you specify a toml file to run multiple deqps from the same runner for better CPU usage and results visualization:
[[deqp]]
deqp = "/home/anholt/src/VK-GL-CTS-build/modules/gles2/deqp-gles2"
caselist = ["/home/anholt/src/VK-GL-CTS-build/external/openglcts/modules/gl_cts/data/mustpass/gles/aosp_mustpass/3.2.5.x/gles2-master.txt"]
baseline = "/home/anholt/src/mesa/src/gallium/drivers/softpipe/ci/deqp-softpipe-fails.txt"
skips = ["/home/anholt/src/mesa/.gitlab-ci/deqp-all-skips.txt", "/home/anholt/src/mesa/src/gallium/drivers/softpipe/ci/deqp-softpipe-skips.txt"]
deqp_args = [
"--deqp-surface-width=256",
"--deqp-surface-height=256",
"--deqp-surface-type=pbuffer",
"--deqp-gl-config-name=rgba8888d24s8ms0",
"--deqp-visibility=hidden"
]
[[deqp]]
deqp = "/home/anholt/src/VK-GL-CTS-build/modules/gles3/deqp-gles3"
caselist = ["/home/anholt/src/VK-GL-CTS-build/external/openglcts/modules/gl_cts/data/mustpass/gles/aosp_mustpass/3.2.5.x/gles3-master.txt"]
baseline = "/home/anholt/src/mesa/src/gallium/drivers/softpipe/ci/deqp-softpipe-fails.txt"
skips = ["/home/anholt/src/mesa/.gitlab-ci/deqp-all-skips.txt", "/home/anholt/src/mesa/src/gallium/drivers/softpipe/ci/deqp-softpipe-skips.txt"]
deqp_args = [
"--deqp-surface-width=256",
"--deqp-surface-height=256",
"--deqp-surface-type=pbuffer",
"--deqp-gl-config-name=rgba8888d24s8ms0",
"--deqp-visibility=hidden"
]
deqp-runner suite --output output/ --testlog-to-xml ~/src/VK-GL-CTS-build/executor/testlog-to-xml --suite ~/cts-softpipe.toml
TODO:
- Do we like the command line interface?
- Are the keys in the right places?
- Should baseline or skips be top-level? Should we do it at both levels like fractions?
- should output dir be configurable per test?
- Add more integration tests?
- Decide if we want to integrate with piglit at all.
- Add test prefixes so we can distinguish the same testcase when run with different config knobs
Edited by Emma Anholt