Executor: The job process sometimes fails to connect to the client in time
++ env PYTHONUNBUFFERED=1 executorctl run -w b2c.yml.jinja2 -j vkcts-vega10-valve -s job_folder
Packing up the share_directory
--> Wrote 43664447 bytes into tar archive...
Waiting for the executor to connect to our local port 37449
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/valve_gfx_ci/executor/client/client.py", line 182, in _setup_connection
sock, _ = tcp_server.accept()
File "/usr/lib64/python3.10/socket.py", line 293, in accept
fd, addr = self._accept()
TimeoutError: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/valve_gfx_ci/executor/client/client.py", line 311, in run
sock, response = self._setup_connection()
File "/usr/local/lib/python3.10/site-packages/valve_gfx_ci/executor/client/client.py", line 184, in _setup_connection
raise ValueError("The server failed to initiate a connection")
ValueError: The server failed to initiate a connection
2022-10-31 13:18:18,398 [INFO] run_job: status: JobStatus.UNKNOWN [MainThread]