How to Ensure Unique Sequential Data Assignment from CSV for Multiple VUs in Artillery?

[pbishung]

Hello Artillery community,

I’m currently working on a performance testing script where I need multiple virtual users (VUs) to consume data from CSV files. The goal is to ensure that:

1. Each VU reads data rows in sequence.
2. The same data row is not reused by other VUs.
3. This behavior works consistently, even when the number of VUs increases.

Here's the relevant part of my current configuration:

Despite setting order: sequence and unique: true, I am concerned about:

Potential overlaps when running high-concurrency scenarios.
Ensuring deterministic behavior, where no two VUs use the same data row.

Use Case:
The test simulates operations like user registration, renewal, and cancellation. Each operation requires distinct data rows (e.g., usernames, VINs) to prevent conflicts during execution.

Questions:

1. Does the combination of order: sequence and unique: true guarantee that each data row is only assigned to one VU?
2. Are there best practices for handling larger datasets to ensure scalability with Artillery?
3. Is there a way to validate or debug how data rows are distributed among VUs?
4. Would external data sources or custom data loading script be a better alternative for managing unique data assignments? If yes, could you provide some guidance?

I’d appreciate insights or examples from anyone who has tackled a similar problem.

Thanks in advance! 🙌

[hassy]

hi @pbishung - this is a very common use-case! we recommend using an external store for this. Redis works really well & we have an example here: https://github.com/artilleryio/artillery/tree/main/examples/using-data-from-redis

Does the combination of order: sequence and unique: true guarantee that each data row is only assigned to one VU?

order: sequence does but only when running Artillery locally and not in distributed mode (e.g. on AWS Lambda/Fargate or Azure ACI)

Are there best practices for handling larger datasets to ensure scalability with Artillery?

Redis is our go-to recommendation for large datasets

Is there a way to validate or debug how data rows are distributed among VUs?

You can log the values assigned to each VU inside the scenario with a log step, something like:

- log: "{{ userName_Registration }}"