Skip to content
This repository was archived by the owner on Feb 27, 2021. It is now read-only.

Commit

Permalink
First implementation
Browse files Browse the repository at this point in the history
  • Loading branch information
ursm committed Apr 1, 2017
0 parents commit efa6304
Show file tree
Hide file tree
Showing 18 changed files with 529 additions and 0 deletions.
12 changes: 12 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
/.bundle/
/.yardoc
/Gemfile.lock
/_yardoc/
/coverage/
/doc/
/pkg/
/spec/reports/
/tmp/

# rspec failure tracking
.rspec_status
2 changes: 2 additions & 0 deletions .rspec
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
--format documentation
--color
10 changes: 10 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
sudo: false
language: ruby

rvm:
- 2.4.1

before_install:
- curl -sL https://dl.google.com/dl/cloudsdk/channels/rapid/google-cloud-sdk.tar.gz | tar xz
- google-cloud-sdk/install.sh --override-components beta pubsub-emulator --quiet
- export PATH=$PATH:$PWD/google-cloud-sdk/bin
5 changes: 5 additions & 0 deletions Gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
source 'https://rubygems.org'

gemspec

gem 'concurrent-ruby-ext'
21 changes: 21 additions & 0 deletions LICENSE.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
The MIT License (MIT)

Copyright (c) 2017 Keita Urashima

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
110 changes: 110 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
# ActiveJob::GoogleCloudPubsub

Google Cloud Pub/Sub adapter and worker for ActiveJob

## Installation

```ruby
gem 'activejob-google_cloud_pubsub'
```

## Usage

First, change the ActiveJob backend.

``` ruby
Rails.application.config.active_job.queue_adapter = :google_cloud_pubsub
```

Write the Job class and code to use it.

``` ruby
class HelloJob < ApplicationJob
def perform(name)
puts "hello, #{name}!"
end
end
```

``` ruby
class HelloController < ApplicationController
def say
HelloJob.perform_later params[:name]
end
end
```

In order to test the worker in your local environment, it is a good idea to use the Pub/Sub emulator provided by `gcloud` command. Refer to [this document](https://cloud.google.com/pubsub/docs/emulator) for the installation procedure.

When the installation is completed, execute the following command to start up the worker.

``` sh
$ gcloud beta emulators pubsub start

(Switch to another terminal)

$ eval `gcloud beta emulators pubsub env-init`
$ cd path/to/your-app
$ bundle exec activejob-google_cloud_pubsub-worker
```

If you hit the previous action, the job will be executed.
(Both the emulator and the worker stop with <kbd>Ctrl+C</kbd>)

## Configuration

### Adapter

When passing options to the adapter, you need to create the object instead of a symbol.

``` ruby
Rails.application.config.active_job.queue_adapter = ActiveJob::QueueAdapters::GoogleCloudPubsubAdapter.new(options)
```

All options are passed to [`Google::Cloud::Pubsub.new`](http://googlecloudplatform.github.io/google-cloud-ruby/#/docs/google-cloud-pubsub/v0.23.2/google/cloud/pubsub?method=new-class).

### Worker

The following command line flags can be specified. All flags are optional.

#### `--require=PATH`

The path of the file to load before the worker starts up.

Default: `./config/environment`

#### `--queue=NAME`

The name of the queue the worker handles.

Note: One worker can handle only one queue. If you use multiple queues, you need to launch multiple worker processes.

Default: `default`

#### `--min_threads=N`

Minimum number of worker threads.

Default: `0`

#### `--max_threads=N`

Maximum number of worker threads.

Default: number of logical cores

#### `--project=PROJECT_ID`, `--keyfile=PATH`

Credentials of Google Cloud Platform. Please refer to [the document](https://github.com/GoogleCloudPlatform/google-cloud-ruby/blob/master/AUTHENTICATION.md) for details.

``` sh
$ bundle exec rake spec
```

## Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/ursm/activejob-google_cloud_pubsub.

## License

The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
6 changes: 6 additions & 0 deletions Rakefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
require 'bundler/gem_tasks'
require 'rspec/core/rake_task'

RSpec::Core::RakeTask.new(:spec)

task :default => :spec
31 changes: 31 additions & 0 deletions activejob-google_cloud_pubsub.gemspec
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
lib = File.expand_path('../lib', __FILE__)
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
require 'active_job/google_cloud_pubsub/version'

Gem::Specification.new do |spec|
spec.name = 'activejob-google_cloud_pubsub'
spec.version = ActiveJob::GoogleCloudPubsub::VERSION
spec.authors = ['Keita Urashima']
spec.email = ['[email protected]']

spec.summary = 'Google Cloud Pub/Sub adapter and worker for ActiveJob'
spec.homepage = 'https://github.com/ursm/activejob-google_cloud_pubsub'
spec.license = 'MIT'

spec.files = `git ls-files -z`.split("\x0").reject do |f|
f.match(%r{^(test|spec|features)/})
end

spec.bindir = 'exe'
spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
spec.require_paths = ['lib']

spec.add_runtime_dependency 'activejob'
spec.add_runtime_dependency 'activesupport'
spec.add_runtime_dependency 'concurrent-ruby'
spec.add_runtime_dependency 'google-cloud-pubsub'

spec.add_development_dependency 'bundler'
spec.add_development_dependency 'rake'
spec.add_development_dependency 'rspec'
end
14 changes: 14 additions & 0 deletions bin/console
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
#!/usr/bin/env ruby

require 'bundler/setup'
require 'activejob-google_cloud_pubsub'

# You can add fixtures and/or initialization code here to make experimenting
# with your gem easier. You can also use a different console, if you like.

# (If you use this, don't forget to add pry to your Gemfile!)
# require "pry"
# Pry.start

require 'irb'
IRB.start(__FILE__)
8 changes: 8 additions & 0 deletions bin/setup
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'
set -vx

bundle install

# Do any other automated setup that you need to do here
41 changes: 41 additions & 0 deletions exe/activejob-google_cloud_pubsub-worker
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
#!/usr/bin/env ruby

require 'activejob-google_cloud_pubsub'
require 'google/cloud/pubsub'
require 'optparse/kwargs'

Version = ActiveJob::GoogleCloudPubsub::VERSION

args = {
require: './config/environment'
}

opt = OptionParser.new

opt.on '--[no-]require=PATH' do |v|
args[:require] = v
end

worker_args = opt.define_by_keywords(
{},
ActiveJob::GoogleCloudPubsub::Worker.instance_method(:initialize),
{
min_threads: Integer,
max_threads: Integer
}
)

pubsub_args = opt.define_by_keywords(
{},
Google::Cloud::Pubsub.method(:new),
{
scope: Array,
timeout: Integer
}
)

opt.parse ARGV

require args[:require] if args[:require]

ActiveJob::GoogleCloudPubsub::Worker.new(**worker_args, **pubsub_args).run
29 changes: 29 additions & 0 deletions lib/active_job/google_cloud_pubsub/adapter.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
require 'active_job/google_cloud_pubsub/naming'
require 'google/cloud/pubsub'
require 'json'

module ActiveJob
module GoogleCloudPubsub
class Adapter
include Naming

def initialize(**pubsub_args)
@pubsub = Google::Cloud::Pubsub.new(pubsub_args)
end

def enqueue(job, attributes = {})
topic = @pubsub.topic(topic_name(job.queue_name), autocreate: true)

topic.publish JSON.dump(job.serialize), attributes
end

def enqueue_at(job, timestamp)
enqueue job, timestamp: timestamp
end
end
end
end

require 'active_job'

ActiveJob::QueueAdapters::GoogleCloudPubsubAdapter = ActiveJob::GoogleCloudPubsub::Adapter
13 changes: 13 additions & 0 deletions lib/active_job/google_cloud_pubsub/naming.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
module ActiveJob
module GoogleCloudPubsub
module Naming
def topic_name(queue_name)
"activejob-queue-#{queue_name}"
end

def subscription_name(queue_name)
"activejob-worker-#{queue_name}"
end
end
end
end
5 changes: 5 additions & 0 deletions lib/active_job/google_cloud_pubsub/version.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
module ActiveJob
module GoogleCloudPubsub
VERSION = '0.1.0'
end
end
Loading

0 comments on commit efa6304

Please sign in to comment.