Product idea

[AndrewDryga]

The Idea

I've seen few companies that build in-house solutions to receive webhooks in a resilient and highly available fashion. This can be archived by buffering all incoming webhooks into queue (:dets, :disc_log, Kafka, RabbitMQ or Faktory) and retrying their delivery to the upstream.

Core concepts

• Business source - free with OS if you want to deploy it to your infrastructure with more features for paid version and access to PRO version source;
• Easy deployment scripts (Docker Container, Heroku, Kubernetes);
• Minimalistic;
• Reliable.

Business model

We can follow https://sidekiq.org/.

Features

Free

• Persistence (must have);
• Delivery retries;
• Web UI;
• Probably debugging tool to investigate what's wrong;

SMB

• Stats (msg Q len and others, check RabbitMQ);
• Debouncing on header/body attribute;
• Ability to collect events and sending them to upstream. (By timeout + messages len.)

Enterprise

• LIFO guarantees (enterprise);
• Rate limiting
• Encryption

Use cases:

• Twilio - it doesn't provide a way to retry failed deliveries, but it's able to redeliver to a secondary endpoint. Simple buffering would solve this issue a lot.
• Events from browsers - when site tracks events which are persisted into internal DB, it's not always available to handle a write-intensive load spike.

Name ideas: buffer, retry it, request me, callbacker