Skip to content

Add test-suite support for deqp-runner.

Emma Anholt requested to merge run-plan into main

Lets you specify a toml file to run multiple deqps from the same runner for better CPU usage and results visualization:

[[deqp]]
deqp = "/home/anholt/src/VK-GL-CTS-build/modules/gles2/deqp-gles2"
caselist = ["/home/anholt/src/VK-GL-CTS-build/external/openglcts/modules/gl_cts/data/mustpass/gles/aosp_mustpass/3.2.5.x/gles2-master.txt"]
baseline = "/home/anholt/src/mesa/src/gallium/drivers/softpipe/ci/deqp-softpipe-fails.txt"
skips = ["/home/anholt/src/mesa/.gitlab-ci/deqp-all-skips.txt", "/home/anholt/src/mesa/src/gallium/drivers/softpipe/ci/deqp-softpipe-skips.txt"]
deqp_args = [
    "--deqp-surface-width=256",
    "--deqp-surface-height=256",
    "--deqp-surface-type=pbuffer",
    "--deqp-gl-config-name=rgba8888d24s8ms0",
    "--deqp-visibility=hidden"
]

[[deqp]]
deqp = "/home/anholt/src/VK-GL-CTS-build/modules/gles3/deqp-gles3"
caselist = ["/home/anholt/src/VK-GL-CTS-build/external/openglcts/modules/gl_cts/data/mustpass/gles/aosp_mustpass/3.2.5.x/gles3-master.txt"]
baseline = "/home/anholt/src/mesa/src/gallium/drivers/softpipe/ci/deqp-softpipe-fails.txt"
skips = ["/home/anholt/src/mesa/.gitlab-ci/deqp-all-skips.txt", "/home/anholt/src/mesa/src/gallium/drivers/softpipe/ci/deqp-softpipe-skips.txt"]
deqp_args = [
    "--deqp-surface-width=256",
    "--deqp-surface-height=256",
    "--deqp-surface-type=pbuffer",
    "--deqp-gl-config-name=rgba8888d24s8ms0",
    "--deqp-visibility=hidden"
]

deqp-runner suite --output output/ --testlog-to-xml ~/src/VK-GL-CTS-build/executor/testlog-to-xml --suite ~/cts-softpipe.toml

TODO:

  • Do we like the command line interface?
  • Are the keys in the right places?
    • Should baseline or skips be top-level? Should we do it at both levels like fractions?
    • should output dir be configurable per test?
  • Add more integration tests?
  • Decide if we want to integrate with piglit at all.
  • Add test prefixes so we can distinguish the same testcase when run with different config knobs
Edited by Emma Anholt

Merge request reports

Loading