Dictionary

General Meanings

Streaming: Continuous and steady data flow between two contexts. Needs one side to generate, another side to receive the data, automatically.

Pipeline: An assembly of stages that are connected with neigbors and run at the same time with them to maximize overall throughput of data flow.

---

Cekirdekler API

Streaming data: zero-copy access between device and host. Used in both multi-gpu and single-gpu.

Event driven pipeline:

• overlaps data-read(of all input arrays) with first kernel execution in the list of kernel names.
• computes all intermediate kernels.
• overlaps data-write(of all output arrays) with last kernel execution in the list of kernel names.

works with multi-single gpu

Driver controlled pipeline:

• divides all work into smaller read+compute+write operations
• sends all concurrently to gpu which gets driver controlled overlapping behavior

works with multi-single gpu

Device to device pipeline:

• only single gpu per pipeline stage is assumed to be used
• data flows through pipeline only 1 stage at a time
• data exits the pipeline after N times
• gpu-compute and gpu-gpu data transitions are overlapped. host-gpu and gpu-host transitions are not overlapped and serialized with gpu compute.
• higher latency than other pipelines, may have higher througput than other pipelines.

Enqueue Mode:

• Meant to optimize single GPU scenarios. Uses single command queue for all work.
• Musch less accumulation of API-overhead over thousands of compute()
• Async mode enables multiple command queues for different compute() groups in a single enqueue mode batch. Overlaps all queues work to save time.
• Used to built custom single-GPU pipelines.