Skip to content

Allow number of fetch workers to be a configurable.

pipeline_results_for_id will indirectly spin up a lot of threads without configuration (20 currently) for fetching resources over HTTP.

If we change the pagination size, the number of threads also can grow in a surprising way. My machine has 40 cores for example, and it was surprising to see 20 Pythons all of sudden :-)

Add a setting to configure this and default it to 4 (arbitrary choice)

Edited by Charlie Turner

Merge request reports

Loading