Grinding the api-queue-worker pattern for fun
- practice PostgreSQL
- practice PostgREST
- practice benchmarking
- practice Rust
Expose an API that can:
- Create a task of a specific type and execution time, returning the task's ID
- Show a list of tasks, filterable by their state (whatever states you define) and/or their task type
- Show a task based on its ID
- Delete a task based on its ID
- The tasks must be persisted into some external data store (your choice).
- Process each task only once and only at/after their specified execution time.
- Support running multiple instances of your code in parallel.
- Queues in Postgres | Postgres.FM 042 | #PostgreSQL #Postgres podcast on YouTube
- Choose Postgres queue technology by Adriano Caloiaro
- PGMQ A lightweight message queue. Like AWS SQS and RSMQ but on Postgres
- River Atomic, transaction-safe, robust job queueing for Go applications. Backed by PostgreSQL and built to scale
-
v0 quick and dirty implementation where main focus was for exploring axum framework and rust multithreading
-
v1 going down the PostgreSQL rabbit hole and some sneak-peaking at this codebases:
-
v2 mostly reusing
v1, incorporating aide for OAS, bundling scheduler, queue and worker -
v3 telemetry, benchmarking and performance improvements
-
v4 new approach inspired by PGMQ but also other additional materials. Some learnings:
- long polling vs short polling trade-offs