This repository was archived by the owner on Feb 27, 2021. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 21
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
0 parents
commit efa6304
Showing
18 changed files
with
529 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
/.bundle/ | ||
/.yardoc | ||
/Gemfile.lock | ||
/_yardoc/ | ||
/coverage/ | ||
/doc/ | ||
/pkg/ | ||
/spec/reports/ | ||
/tmp/ | ||
|
||
# rspec failure tracking | ||
.rspec_status |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
--format documentation | ||
--color |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
sudo: false | ||
language: ruby | ||
|
||
rvm: | ||
- 2.4.1 | ||
|
||
before_install: | ||
- curl -sL https://dl.google.com/dl/cloudsdk/channels/rapid/google-cloud-sdk.tar.gz | tar xz | ||
- google-cloud-sdk/install.sh --override-components beta pubsub-emulator --quiet | ||
- export PATH=$PATH:$PWD/google-cloud-sdk/bin |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
source 'https://rubygems.org' | ||
|
||
gemspec | ||
|
||
gem 'concurrent-ruby-ext' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
The MIT License (MIT) | ||
|
||
Copyright (c) 2017 Keita Urashima | ||
|
||
Permission is hereby granted, free of charge, to any person obtaining a copy | ||
of this software and associated documentation files (the "Software"), to deal | ||
in the Software without restriction, including without limitation the rights | ||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
copies of the Software, and to permit persons to whom the Software is | ||
furnished to do so, subject to the following conditions: | ||
|
||
The above copyright notice and this permission notice shall be included in | ||
all copies or substantial portions of the Software. | ||
|
||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN | ||
THE SOFTWARE. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,110 @@ | ||
# ActiveJob::GoogleCloudPubsub | ||
|
||
Google Cloud Pub/Sub adapter and worker for ActiveJob | ||
|
||
## Installation | ||
|
||
```ruby | ||
gem 'activejob-google_cloud_pubsub' | ||
``` | ||
|
||
## Usage | ||
|
||
First, change the ActiveJob backend. | ||
|
||
``` ruby | ||
Rails.application.config.active_job.queue_adapter = :google_cloud_pubsub | ||
``` | ||
|
||
Write the Job class and code to use it. | ||
|
||
``` ruby | ||
class HelloJob < ApplicationJob | ||
def perform(name) | ||
puts "hello, #{name}!" | ||
end | ||
end | ||
``` | ||
|
||
``` ruby | ||
class HelloController < ApplicationController | ||
def say | ||
HelloJob.perform_later params[:name] | ||
end | ||
end | ||
``` | ||
|
||
In order to test the worker in your local environment, it is a good idea to use the Pub/Sub emulator provided by `gcloud` command. Refer to [this document](https://cloud.google.com/pubsub/docs/emulator) for the installation procedure. | ||
|
||
When the installation is completed, execute the following command to start up the worker. | ||
|
||
``` sh | ||
$ gcloud beta emulators pubsub start | ||
|
||
(Switch to another terminal) | ||
|
||
$ eval `gcloud beta emulators pubsub env-init` | ||
$ cd path/to/your-app | ||
$ bundle exec activejob-google_cloud_pubsub-worker | ||
``` | ||
|
||
If you hit the previous action, the job will be executed. | ||
(Both the emulator and the worker stop with <kbd>Ctrl+C</kbd>) | ||
|
||
## Configuration | ||
|
||
### Adapter | ||
|
||
When passing options to the adapter, you need to create the object instead of a symbol. | ||
|
||
``` ruby | ||
Rails.application.config.active_job.queue_adapter = ActiveJob::QueueAdapters::GoogleCloudPubsubAdapter.new(options) | ||
``` | ||
|
||
All options are passed to [`Google::Cloud::Pubsub.new`](http://googlecloudplatform.github.io/google-cloud-ruby/#/docs/google-cloud-pubsub/v0.23.2/google/cloud/pubsub?method=new-class). | ||
|
||
### Worker | ||
|
||
The following command line flags can be specified. All flags are optional. | ||
|
||
#### `--require=PATH` | ||
|
||
The path of the file to load before the worker starts up. | ||
|
||
Default: `./config/environment` | ||
|
||
#### `--queue=NAME` | ||
|
||
The name of the queue the worker handles. | ||
|
||
Note: One worker can handle only one queue. If you use multiple queues, you need to launch multiple worker processes. | ||
|
||
Default: `default` | ||
|
||
#### `--min_threads=N` | ||
|
||
Minimum number of worker threads. | ||
|
||
Default: `0` | ||
|
||
#### `--max_threads=N` | ||
|
||
Maximum number of worker threads. | ||
|
||
Default: number of logical cores | ||
|
||
#### `--project=PROJECT_ID`, `--keyfile=PATH` | ||
|
||
Credentials of Google Cloud Platform. Please refer to [the document](https://github.com/GoogleCloudPlatform/google-cloud-ruby/blob/master/AUTHENTICATION.md) for details. | ||
|
||
``` sh | ||
$ bundle exec rake spec | ||
``` | ||
|
||
## Contributing | ||
|
||
Bug reports and pull requests are welcome on GitHub at https://github.com/ursm/activejob-google_cloud_pubsub. | ||
|
||
## License | ||
|
||
The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
require 'bundler/gem_tasks' | ||
require 'rspec/core/rake_task' | ||
|
||
RSpec::Core::RakeTask.new(:spec) | ||
|
||
task :default => :spec |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
lib = File.expand_path('../lib', __FILE__) | ||
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib) | ||
require 'active_job/google_cloud_pubsub/version' | ||
|
||
Gem::Specification.new do |spec| | ||
spec.name = 'activejob-google_cloud_pubsub' | ||
spec.version = ActiveJob::GoogleCloudPubsub::VERSION | ||
spec.authors = ['Keita Urashima'] | ||
spec.email = ['[email protected]'] | ||
|
||
spec.summary = 'Google Cloud Pub/Sub adapter and worker for ActiveJob' | ||
spec.homepage = 'https://github.com/ursm/activejob-google_cloud_pubsub' | ||
spec.license = 'MIT' | ||
|
||
spec.files = `git ls-files -z`.split("\x0").reject do |f| | ||
f.match(%r{^(test|spec|features)/}) | ||
end | ||
|
||
spec.bindir = 'exe' | ||
spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) } | ||
spec.require_paths = ['lib'] | ||
|
||
spec.add_runtime_dependency 'activejob' | ||
spec.add_runtime_dependency 'activesupport' | ||
spec.add_runtime_dependency 'concurrent-ruby' | ||
spec.add_runtime_dependency 'google-cloud-pubsub' | ||
|
||
spec.add_development_dependency 'bundler' | ||
spec.add_development_dependency 'rake' | ||
spec.add_development_dependency 'rspec' | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
#!/usr/bin/env ruby | ||
|
||
require 'bundler/setup' | ||
require 'activejob-google_cloud_pubsub' | ||
|
||
# You can add fixtures and/or initialization code here to make experimenting | ||
# with your gem easier. You can also use a different console, if you like. | ||
|
||
# (If you use this, don't forget to add pry to your Gemfile!) | ||
# require "pry" | ||
# Pry.start | ||
|
||
require 'irb' | ||
IRB.start(__FILE__) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
#!/usr/bin/env bash | ||
set -euo pipefail | ||
IFS=$'\n\t' | ||
set -vx | ||
|
||
bundle install | ||
|
||
# Do any other automated setup that you need to do here |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
#!/usr/bin/env ruby | ||
|
||
require 'activejob-google_cloud_pubsub' | ||
require 'google/cloud/pubsub' | ||
require 'optparse/kwargs' | ||
|
||
Version = ActiveJob::GoogleCloudPubsub::VERSION | ||
|
||
args = { | ||
require: './config/environment' | ||
} | ||
|
||
opt = OptionParser.new | ||
|
||
opt.on '--[no-]require=PATH' do |v| | ||
args[:require] = v | ||
end | ||
|
||
worker_args = opt.define_by_keywords( | ||
{}, | ||
ActiveJob::GoogleCloudPubsub::Worker.instance_method(:initialize), | ||
{ | ||
min_threads: Integer, | ||
max_threads: Integer | ||
} | ||
) | ||
|
||
pubsub_args = opt.define_by_keywords( | ||
{}, | ||
Google::Cloud::Pubsub.method(:new), | ||
{ | ||
scope: Array, | ||
timeout: Integer | ||
} | ||
) | ||
|
||
opt.parse ARGV | ||
|
||
require args[:require] if args[:require] | ||
|
||
ActiveJob::GoogleCloudPubsub::Worker.new(**worker_args, **pubsub_args).run |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
require 'active_job/google_cloud_pubsub/naming' | ||
require 'google/cloud/pubsub' | ||
require 'json' | ||
|
||
module ActiveJob | ||
module GoogleCloudPubsub | ||
class Adapter | ||
include Naming | ||
|
||
def initialize(**pubsub_args) | ||
@pubsub = Google::Cloud::Pubsub.new(pubsub_args) | ||
end | ||
|
||
def enqueue(job, attributes = {}) | ||
topic = @pubsub.topic(topic_name(job.queue_name), autocreate: true) | ||
|
||
topic.publish JSON.dump(job.serialize), attributes | ||
end | ||
|
||
def enqueue_at(job, timestamp) | ||
enqueue job, timestamp: timestamp | ||
end | ||
end | ||
end | ||
end | ||
|
||
require 'active_job' | ||
|
||
ActiveJob::QueueAdapters::GoogleCloudPubsubAdapter = ActiveJob::GoogleCloudPubsub::Adapter |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
module ActiveJob | ||
module GoogleCloudPubsub | ||
module Naming | ||
def topic_name(queue_name) | ||
"activejob-queue-#{queue_name}" | ||
end | ||
|
||
def subscription_name(queue_name) | ||
"activejob-worker-#{queue_name}" | ||
end | ||
end | ||
end | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
module ActiveJob | ||
module GoogleCloudPubsub | ||
VERSION = '0.1.0' | ||
end | ||
end |
Oops, something went wrong.