Improve CollatePipeline concurrency, thread safety and type hints
The current CollatePipeline.__job_selection
is thread unsafe and
has complicated type hints.
Split the job filtering functionality in to a different function
_iter_jobs
. When a caller only wants the jobs this new function
can be used. This eliminates the need for typing.overload
.
The __job_selection
is NOT thread safe because it shares args_dict
between the threads and each thread writes its own job_id
to it.
This means it is possible for job_id
to be modified while other
thread still uses it.
Instead make this map function to take a callable which should accept
the CollateJob
and int
(job_id) as arguments. This makes type
hints more accurate as the return type of passed callable can be
made generic meaning the return type will change based on passed
callable.
Make retried jobs requests run concuirrently. There was a TODO in the old code about making it concurrent.
Also rename the __job_selection
to _map_jobs
as that better
describes its functionality.