Skip to content

results_importer: batch queries for finding known tests

clayton craft requested to merge craftyguy/mesa_ci_results:perf into master

This changes the way that known tests are found by 1) generating a list of all tests in the subgroup, 2) selecting tests known tests from sql by grouping creating a single select statement for every 1000 tests, and 3) diffing the two lists to return a list of unknown, or new, tests.

Profiling results_importer revealed that querying sql once per test was causing a lot of latency. My limited testing of imports shows that this change results in a ~2x speed-up, without a substantial increase in memory usage (it was negligible/undetectable on my 16GB laptop).

Merge request reports