v3.5 Recycle Mode

Introduction

Some cases require to perform the load operations on the limited set of objects/files in the unlimited manner (infinitely either using time limit).

Limitations

1. Load type is one of
   • Noop
   • Read
   • Update
2. Items input is configured
3. Items input supplies the items which count is no more than the configured recycle queue size

Requirements

1. Recycle the load operations on the items.
2. Sustain the initial items input order.
3. Respect the operation weight in case of the weighted load.
4. Respect the rate limit if configured.

Approach

The Load Generator component is responsible for the tasks recycling. There are two additional methods in its interface to support the recycling:

• Check if the load generator is configured to recycle the tasks:

  boolean isRecycling();

• Enqueue the load task for further recycling:

  void recycle(final O ioTask);

Specific Recycle Flow

1. Load Generator:
   1. Produces the new load tasks which count is no more than the configured recycle queue size
2. Storage Driver:
   1. Resets the next load task state (status, timestamps, etc)
   2. Executes the next load task
   3. Outputs the next completed load task to the Load Controller
3. Load Controller:
   1. Receives the next completed load task
   2. Drops the load task if its status is not successful
   3. Determines the load task origin (the Load Generator produced this load task)
   4. Checks if the resolved Load Generator is in the recycling mode
   5. Updates the latest results for the corresponding item.
   6. Enqueues the load task back to the Load Generator for further recycle
4. Load Generator:
   1. Begins to recycle only if items input is exhausted
   2. Produces the recycling load tasks from the recycle queue

Specific Load Controller States

1. "Nothing to recycle" state.

Configuration

1. load-generator-recycle The flag to enable the recycle mode. Disabled by default.
2. load-queue-size The recycle queue size. Note that this queue size is also used by storage drivers internal queues.

Output