Skip to content

Latest commit

 

History

History
39 lines (22 loc) · 1.57 KB

README.md

File metadata and controls

39 lines (22 loc) · 1.57 KB

Redis Queue

stoQ plugin that interacts with Google Cloud Pub/Sub for queuing

Plugin Classes

Configuration

All options below may be set by:

Options

  • project_id [str]: Google Cloud project ID

  • topic [str]: Pub/Sub Topic to bind to

  • subscription [str]: Pub/Sub Subscription to monitor for messages

  • max_messages [int]: Maximum number of messages to pull at once

Usage

Pub/Sub Queuing Example

Start stoq using pubsub as the provider plugin, filedir as the source archiver, then scan the payload with the hash, and send results to stdout connector:

$ stoq run -P pubsub -S filedir -a hash -C stdout

In another terminal, load files from /tmp/test-files using the filedir provider plugin, then archive the payloads with filedir archive plugin, and send a message to the pubsub queue for processing:

$ stoq run -P filedir -A filedir -C pubsub --plugin-opts filedir:source_dir=/tmp/test-files