Load Sources/Targets adapters in parallel

[chadell]

Proposed Functionality

Load Source and Targets concurrently

Use Case

Loading adapters, being I/O operations that have no dependencies between them, could be run in parallel and save time

[jdrew82]

I think this might need to be done directly in DiffSync and not in this plugin. I'd imagine we'd need to make it asynchronous but I'm not sure what the implications are for that change on the rest of the ecosystem.

[chadell]

Diffsync support the functionality with the new Redis backend, but it has to be called from the plugin. The idea is to apply data pipeline approach here (I have a PoC working).

[jdrew82]

Interesting. Would love to see this in operation. Would be very helpful in these large dataset imports.

[itdependsnetworks]

I created this issue on diffsync, not realizing this was here: networktocode/diffsync#202

[chadell]

We decided to put this on hold until Nautobot 2.0 is out with new Job features and other implementations using external data pipelines are evaluated