Skip to content

perf_builder: handle crashed benchmark iteration

clayton craft requested to merge update into master

This catches a benchmark crash, checks for gpu hangs, and bails out discarding any results for previous iteration of the benchmark.

This is added because there are some cases where the benchmark crashes from a gpu hang, but CI continues on and 'passes' the shard without ever calling the check for gpu hangs. The next time something runs on the system, CI detects GPU hang and bails out without running anything and schedules a reboot. This causes gaps in the results for that build.

Merge request reports