diff --git a/README.md b/README.md index 2f976f7..1e029f5 100644 --- a/README.md +++ b/README.md @@ -4,13 +4,13 @@ Create Microservices in Rails by pretty much just writing regular Rails code. This gem provides: -* transparent API key authentication. -* router-level API version based on headers. -* a way to document your microservice endpoints via acceptance tests. -* structured errors, buildable from invalid Active Records, Exceptions, or by hand. +- transparent API key authentication. +- router-level API version based on headers. +- a way to document your microservice endpoints via acceptance tests. +- structured errors, buildable from invalid Active Records, Exceptions, or by hand. This, plus much of what you get from Rails already, means you can create a microservice Rails application by just writing the -same Rails code you write today. Instead of rendering web views, you render JSON (which is built into Rails). +same Rails code you write today. Instead of rendering web views, you render JSON (which is built into Rails). ## To install @@ -35,14 +35,26 @@ Then, set it up: ### Upgrading from an older version -* If you have a version lower than 3.3.0, you need to run two generators, one of which creates a new database migration on your -`api_clients` table: +- When upgrading to version 4.0.0 you may now take advantage of an in-memory cache + +You can enabled it like so + +```ruby +Stitches.configure do |config| + config.max_cache_ttl = 5 # seconds + config.max_cache_size = 100 # how many keys to cache +end +``` + +- If you have a version lower than 3.3.0, you need to run two generators, one of which creates a new database migration on your + `api_clients` table: ``` > bin/rails generate stitches:add_enabled_to_api_clients > bin/rails generate stitches:add_deprecation ``` -* If you have a version lower than 3.6.0, you need to run one generator: + +- If you have a version lower than 3.6.0, you need to run one generator: ``` > bin/rails generate stitches:add_deprecation @@ -59,8 +71,8 @@ class Api::V1::WidgetsController < ApiController if widget.valid? head 201 else - render json: { - errors: Stitches::Errors.from_active_record_object(widget) + render json: { + errors: Stitches::Errors.from_active_record_object(widget) }, status: 422 end end @@ -73,44 +85,62 @@ private end ``` -If you think there's nothing special about this—you are correct. This is the vanillaest of vanilla Rails controllers, with a few +If you think there's nothing special about this—you are correct. This is the vanillaest of vanilla Rails controllers, with a few notable exceptions: -* We aren't checking content type. A stitches-based microservice always uses JSON and refuses to route requests for non-JSON to -you, so there's zero need to use `respond_to` and friends. -* The error-building is structured and reliable. -* This is an authenticated request. No request without proper authentication will be routed here, so you don't have to worry -about it in your code. -* This is a versioned request. While the URL will *not* contain `v1` in it, the `Accept` header will require a version and get -routed here. If you make a V2, it's just a new controller and this concern is handled at the routing layer. +- We aren't checking content type. A stitches-based microservice always uses JSON and refuses to route requests for non-JSON to + you, so there's zero need to use `respond_to` and friends. +- The error-building is structured and reliable. +- This is an authenticated request. No request without proper authentication will be routed here, so you don't have to worry + about it in your code. +- This is a versioned request. While the URL will _not_ contain `v1` in it, the `Accept` header will require a version and get + routed here. If you make a V2, it's just a new controller and this concern is handled at the routing layer. -All this means that the Rails skills of you and your team can be directly applied to building microservices. You don't have to make a bunch of boring decisions about auth, versioning, or content-types. It also means you can start deploying and creating microservices with little friction. No need to deal with a complex DSL or new programming language to get yourselves going with Microservices. +All this means that the Rails skills of you and your team can be directly applied to building microservices. You don't have to make a bunch of boring decisions about auth, versioning, or content-types. It also means you can start deploying and creating microservices with little friction. No need to deal with a complex DSL or new programming language to get yourselves going with Microservices. ## More Info See [the wiki](https://github.com/stitchfix/stitches/wiki/Setup) for how to setup stitches. -* [Stitches Features](https://github.com/stitchfix/stitches/wiki/Features-of-Stitches) include: +- [Stitches Features](https://github.com/stitchfix/stitches/wiki/Features-of-Stitches) include: - Authorization via API key - Versioned requests via HTTP content types - Structured Errors - ISO 8601-formatted dates - Deprecation using the `Sunset` header -* The [Generator](https://github.com/stitchfix/stitches/wiki/Generator) sets up some code in your app, so you can start writing -APIs using vanilla Rails idioms: + - An optional ApiKey cache to allow mostly DB free APIs +- The [Generator](https://github.com/stitchfix/stitches/wiki/Generator) sets up some code in your app, so you can start writing + APIs using vanilla Rails idioms: - a "ping" controller that can validate your app is working - version routing based on content-type (requests for V2 use the same URL, but are serviced by a different controller) - An ApiClient Active Record - Acceptance tests that can produce API documentation as they test your app. -* Stitches provides [testing support](https://github.com/stitchfix/stitches/wiki/Testing) +- Stitches provides [testing support](https://github.com/stitchfix/stitches/wiki/Testing) + +## API Key Caching + +Since version 4.0.0, stitches now has the ability to cache API keys in +memory for a configurable amount of time. This may be an improvement for +some applications. + +You must configure the API Cache for it be used. + +```ruby +Stitches.configure do |config| + config.max_cache_ttl = 5 # seconds + config.max_cache_size = 100 # how many keys to cache +end +``` +Your cache size should be +larger then the number of consumer keys your service has. ## Developing -Although `Stitches.configuration` is global, do not depend directly on that in your logic. Instead, allow all classes to receive a configuration object in their constructor. This makes the classes easier to deal with and change, without incurring much of a real cost to development. Global symbols suck, but are convenient. This is how you make the most of it. +Although `Stitches.configuration` is global, do not depend directly on that in your logic. Instead, allow all classes to receive a configuration object in their constructor. This makes the classes easier to deal with and change, without incurring much of a real cost to development. Global symbols suck, but are convenient. This is how you make the most of it. Also, the integration test does a lot of "testing the implementation", but since Rails generators are notorious for silently -failing with a successful result, we have to make sure that the various `inject_into_file` calls are actually working. Do not do +failing with a successful result, we have to make sure that the various `inject_into_file` calls are actually working. Do not do any fancy refactors here, just keep it up to date. --- diff --git a/lib/stitches/allowlist_middleware.rb b/lib/stitches/allowlist_middleware.rb index b072436..99d60b6 100644 --- a/lib/stitches/allowlist_middleware.rb +++ b/lib/stitches/allowlist_middleware.rb @@ -2,7 +2,6 @@ module Stitches # A middleware that will skip its behavior if the path matches an allowed URL class AllowlistMiddleware def initialize(app, options={}) - @app = app @configuration = options[:configuration] || Stitches.configuration @except = options[:except] || @configuration.allowlist_regexp diff --git a/lib/stitches/api_client_access_wrapper.rb b/lib/stitches/api_client_access_wrapper.rb new file mode 100644 index 0000000..a5116d1 --- /dev/null +++ b/lib/stitches/api_client_access_wrapper.rb @@ -0,0 +1,42 @@ +require 'lru_redux' + +module Stitches::ApiClientAccessWrapper + + def self.fetch_for_key(key) + if cache_enabled + fetch_for_key_from_cache(key) + else + fetch_for_key_from_db(key) + end + end + + def self.fetch_for_key_from_cache(key) + api_key_cache.getset(key) do + fetch_for_key_from_db(key) + end + end + + def self.fetch_for_key_from_db(key) + if ::ApiClient.column_names.include?("enabled") + ::ApiClient.find_by(key: key, enabled: true) + else + ActiveSupport::Deprecation.warn('api_keys is missing "enabled" column. Run "rails g stitches:add_enabled_to_api_clients"') + ::ApiClient.find_by(key: key) + end + end + + def self.clear_api_cache + api_key_cache.clear if cache_enabled + end + + def self.api_key_cache + @api_key_cache ||= LruRedux::TTL::ThreadSafeCache.new( + Stitches.configuration.max_cache_size, + Stitches.configuration.max_cache_ttl, + ) + end + + def self.cache_enabled + Stitches.configuration.max_cache_ttl.positive? + end +end \ No newline at end of file diff --git a/lib/stitches/api_key.rb b/lib/stitches/api_key.rb index b5631f9..843d13d 100644 --- a/lib/stitches/api_key.rb +++ b/lib/stitches/api_key.rb @@ -27,14 +27,7 @@ def do_call(env) if authorization if authorization =~ /#{@configuration.custom_http_auth_scheme}\s+key=(.*)\s*$/ key = $1 - - if ApiClient.column_names.include?("enabled") - client = ApiClient.where(key: key, enabled: true).first - else - ActiveSupport::Deprecation.warn('api_keys is missing "enabled" column. Run "rails g stitches:add_enabled_to_api_clients"') - client = ApiClient.where(key: key).first - end - + client = Stitches::ApiClientAccessWrapper.fetch_for_key(key) if client.present? env[@configuration.env_var_to_hold_api_client_primary_key] = client.id env[@configuration.env_var_to_hold_api_client] = client diff --git a/lib/stitches/configuration.rb b/lib/stitches/configuration.rb index 5ed4425..5e8603d 100644 --- a/lib/stitches/configuration.rb +++ b/lib/stitches/configuration.rb @@ -13,6 +13,8 @@ def reset_to_defaults! @custom_http_auth_scheme = UnsetString.new("custom_http_auth_scheme") @env_var_to_hold_api_client_primary_key = NonNullString.new("env_var_to_hold_api_client_primary_key","STITCHES_API_CLIENT_ID") @env_var_to_hold_api_client= NonNullString.new("env_var_to_hold_api_client","STITCHES_API_CLIENT") + @max_cache_ttl = NonNullInteger.new("max_cache_ttl", 0) + @max_cache_size = NonNullInteger.new("max_cache_size", 0) end # A RegExp that allows URLS around the mime type and api key requirements. @@ -25,11 +27,6 @@ def allowlist_regexp=(new_allowlist_regexp) @allowlist_regexp = new_allowlist_regexp end - def whitelist_regexp=(new_allowlist_regexp) - self.allowlist_regexp = new_allowlist_regexp - warn("⚠️ 'whitelist' is deprecated in stitches configuration, please use 'allowlist' or auto-update with:\n\n bin/rails g stitches:update_configuration\n\n⚠️ 'whitelist' will be removed in 4.0") - end - # The name of your custom http auth scheme. This must be set, and has no default def custom_http_auth_scheme @custom_http_auth_scheme.to_s @@ -39,7 +36,7 @@ def custom_http_auth_scheme=(new_custom_http_auth_scheme) @custom_http_auth_scheme = NonNullString.new("custom_http_auth_scheme",new_custom_http_auth_scheme) end - # The name of the environment variable that the ApiKey middleware should use to + # The name of the environment variable that the ApiKey middleware should use to # place the primary key of the authenticated ApiKey. For example, if a user provides # the api key 1234-1234-1234-1234, and that maps to the primary key 42 in your database, # the environment will contain "42" in the key provided here. @@ -59,8 +56,40 @@ def env_var_to_hold_api_client=(new_env_var_to_hold_api_client) @env_var_to_hold_api_client= NonNullString.new("env_var_to_hold_api_client",new_env_var_to_hold_api_client) end + def max_cache_ttl + @max_cache_ttl.to_i + end + + def max_cache_ttl=(new_max_cache_ttl) + @max_cache_ttl = NonNullInteger.new("max_cache_ttl", new_max_cache_ttl) + end + + def max_cache_size + @max_cache_size.to_i + end + + def max_cache_size=(new_max_cache_size) + @max_cache_size = NonNullInteger.new("max_cache_size", new_max_cache_size) + end + private + class NonNullInteger + def initialize(name, value) + unless value.is_a?(Integer) + raise "#{name} must be an Integer, not a #{value.class}" + end + + @value = value + end + + def to_i + @value + end + + alias to_integer to_i + end + class NonNullString def initialize(name,string) unless string.nil? || string.is_a?(String) diff --git a/lib/stitches/generator_files/config/initializers/stitches.rb b/lib/stitches/generator_files/config/initializers/stitches.rb index e2b8f6c..f437ea3 100644 --- a/lib/stitches/generator_files/config/initializers/stitches.rb +++ b/lib/stitches/generator_files/config/initializers/stitches.rb @@ -11,4 +11,14 @@ # Env var that gets the primary key of the authenticated ApiKey # for access in your controllers, so they don't need to re-parse the header # configuration.env_var_to_hold_api_client_primary_key = "YOUR_ENV_VAR" + + # Configures how long to cache ApiKeys in memory (In Seconds) + # A value of 0 will disable the cache entierly + # Default is 0 + # configuration.max_cache_ttl = 5 + + # Configures how many ApiKeys to cache at one time + # This should be larger then the number of clients + # Default is 0 + # configuration.max_cache_size = 100 end diff --git a/lib/stitches/railtie.rb b/lib/stitches/railtie.rb index 5d7e3bf..897c02b 100644 --- a/lib/stitches/railtie.rb +++ b/lib/stitches/railtie.rb @@ -1,9 +1,11 @@ require 'stitches/api_key' require 'stitches/valid_mime_type' +require 'stitches/api_client_access_wrapper' module Stitches class Railtie < Rails::Railtie config.app_middleware.use Stitches::ApiKey config.app_middleware.use Stitches::ValidMimeType + end end diff --git a/lib/stitches/update_configuration_generator.rb b/lib/stitches/update_configuration_generator.rb deleted file mode 100644 index 06e97fd..0000000 --- a/lib/stitches/update_configuration_generator.rb +++ /dev/null @@ -1,16 +0,0 @@ -require 'rails/generators' - -module Stitches - class UpdateConfigurationGenerator < Rails::Generators::Base - include Rails::Generators::Migration - - source_root(File.expand_path(File.join(File.dirname(__FILE__),"generator_files"))) - - desc "Change your configuration to use 'allowlist' so you'll be ready for 4.x" - def update_to_allowlist - gsub_file "config/initializers/stitches.rb", /whitelist/, "allowlist" - puts "🎉 You are now good to go!" - end - - end -end diff --git a/lib/stitches/version.rb b/lib/stitches/version.rb index a680ad3..a628b60 100644 --- a/lib/stitches/version.rb +++ b/lib/stitches/version.rb @@ -1,3 +1,5 @@ +# frozen_string_literal: true + module Stitches - VERSION = "3.8.3" + VERSION = '4.0.0' end diff --git a/lib/stitches/whitelisting_middleware.rb b/lib/stitches/whitelisting_middleware.rb deleted file mode 100644 index 5ff16cd..0000000 --- a/lib/stitches/whitelisting_middleware.rb +++ /dev/null @@ -1,5 +0,0 @@ -require_relative "allowlist_middleware" - -module Stitches - WhitelistingMiddleware = AllowlistMiddleware -end diff --git a/lib/stitches_norailtie.rb b/lib/stitches_norailtie.rb index c39acad..5bb8174 100644 --- a/lib/stitches_norailtie.rb +++ b/lib/stitches_norailtie.rb @@ -14,7 +14,6 @@ def self.configuration require 'stitches/api_generator' require 'stitches/add_deprecation_generator' require 'stitches/add_enabled_to_api_clients_generator' -require 'stitches/update_configuration_generator' require 'stitches/api_version_constraint' require 'stitches/api_key' require 'stitches/deprecation' diff --git a/spec/api_client_access_wrapper_spec.rb b/spec/api_client_access_wrapper_spec.rb new file mode 100644 index 0000000..a8218d6 --- /dev/null +++ b/spec/api_client_access_wrapper_spec.rb @@ -0,0 +1,52 @@ +require 'spec_helper.rb' + +module MyApp + class Application + end +end + +unless defined? ApiClient + class ApiClient + def self.column_names + ["enabled"] + end + end +end + +describe Stitches::ApiClientAccessWrapper do + let(:api_client) { + double(ApiClient, id: 42) + } + before do + Stitches.configuration.reset_to_defaults! + end + describe '#fetch_by_key' do + context "cache is disabled" do + before do + expect(ApiClient).to receive(:find_by).and_return(api_client).twice + end + + it "fetchs object from db twice" do + expect(described_class.fetch_for_key("123").id).to eq(42) + expect(described_class.fetch_for_key("123").id).to eq(42) + end + end + + context "cache is configured" do + before do + Stitches.configure do |config| + config.max_cache_ttl = 5 + config.max_cache_size = 10 + end + + expect(ApiClient).to receive(:find_by).and_return(api_client).once + end + + it "fetchs object from cache" do + expect(described_class.fetch_for_key("123").id).to eq(42) + # This should hit the cache + expect(described_class.fetch_for_key("123").id).to eq(42) + end + end + end +end diff --git a/spec/api_key_spec.rb b/spec/api_key_spec.rb index 7928104..652557f 100644 --- a/spec/api_key_spec.rb +++ b/spec/api_key_spec.rb @@ -15,10 +15,8 @@ def self.column_names describe Stitches::ApiKey do let(:app) { double("rack app") } - let(:api_clients) { - [ - double(ApiClient, id: 42) - ] + let(:api_client) { + double(ApiClient, id: 42) } before do @@ -27,7 +25,8 @@ def self.column_names fake_rails_app = MyApp::Application.new allow(Rails).to receive(:application).and_return(fake_rails_app) allow(app).to receive(:call).with(env) - allow(ApiClient).to receive(:where).and_return(api_clients) + allow(ApiClient).to receive(:find_by).and_return(api_client) + Stitches::ApiClientAccessWrapper.clear_api_cache end subject(:middleware) { described_class.new(app, namespace: "/api") } @@ -158,11 +157,11 @@ def self.column_names end it "sets the api_client's ID in the environment" do - expect(env[Stitches.configuration.env_var_to_hold_api_client_primary_key]).to eq(api_clients.first.id) + expect(env[Stitches.configuration.env_var_to_hold_api_client_primary_key]).to eq(api_client.id) end it "sets the api_client itself in the environment" do - expect(env[Stitches.configuration.env_var_to_hold_api_client]).to eq(api_clients.first) + expect(env[Stitches.configuration.env_var_to_hold_api_client]).to eq(api_client) end end @@ -177,7 +176,7 @@ def self.column_names "HTTP_AUTHORIZATION" => "MyAwesomeInternalScheme key=foobar", } } - let(:api_clients) { [] } + let(:api_client) { nil } it_behaves_like "an unauthorized response" do let(:expected_body) { "Unauthorized - key invalid" } diff --git a/spec/configuration_spec.rb b/spec/configuration_spec.rb index 0f4e210..d5c4368 100644 --- a/spec/configuration_spec.rb +++ b/spec/configuration_spec.rb @@ -9,17 +9,23 @@ let(:allowlist_regexp) { %r{foo} } let(:custom_http_auth_scheme) { "Blah" } let(:env_var_to_hold_api_client_primary_key) { "FOOBAR" } + let(:max_cache_ttl) { 11 } + let(:max_cache_size) { 111 } it "can be configured globally" do Stitches.configure do |config| config.allowlist_regexp = allowlist_regexp config.custom_http_auth_scheme = custom_http_auth_scheme config.env_var_to_hold_api_client_primary_key = env_var_to_hold_api_client_primary_key + config.max_cache_ttl = max_cache_ttl + config.max_cache_size = max_cache_size end expect(Stitches.configuration.allowlist_regexp).to eq(allowlist_regexp) expect(Stitches.configuration.custom_http_auth_scheme).to eq(custom_http_auth_scheme) expect(Stitches.configuration.env_var_to_hold_api_client_primary_key).to eq(env_var_to_hold_api_client_primary_key) + expect(Stitches.configuration.max_cache_ttl).to eq(max_cache_ttl) + expect(Stitches.configuration.max_cache_size).to eq(max_cache_size) end it "defaults to nil for allowlist_regexp" do @@ -30,6 +36,14 @@ expect(Stitches.configuration.env_var_to_hold_api_client_primary_key).to eq("STITCHES_API_CLIENT_ID") end + it "defaults to 0 for max_cache_ttl" do + expect(Stitches.configuration.max_cache_ttl).to eq(0) + end + + it "sets a default for max_cache_size" do + expect(Stitches.configuration.max_cache_size).to eq(0) + end + it "blows up if you try to use custom_http_auth_scheme without having set it" do expect { Stitches.configuration.custom_http_auth_scheme @@ -102,19 +116,34 @@ }.not_to raise_error end end - context "deprecated options we want to support for backwards compatibility" do - let(:logger) { double("logger") } - before do - allow(Rails).to receive(:logger).and_return(logger) - allow(logger).to receive(:info) + describe "max_cache_ttl" do + let(:config) { Stitches::Configuration.new } + it "must be an integer" do + expect { + config.max_cache_ttl = "" + }.to raise_error(/max_cache_ttl must be an Integer, not a String/) end - it "'whitelist' still works for allowlist" do - Stitches.configure do |config| - config.whitelist_regexp = /foo/ - end - expect(Stitches.configuration.allowlist_regexp).to eq(/foo/) + it "may not be nil" do + expect { + config.max_cache_ttl = nil + }.to raise_error(/max_cache_ttl must be an Integer, not a NilClass/) + end + end + + describe "max_cache_size" do + let(:config) { Stitches::Configuration.new } + it "must be an integer" do + expect { + config.max_cache_size = "" + }.to raise_error(/max_cache_size must be an Integer, not a String/) + end + + it "may not be nil" do + expect { + config.max_cache_size = nil + }.to raise_error(/max_cache_size must be an Integer, not a NilClass/) end end end diff --git a/spec/integration/add_to_rails_app_spec.rb b/spec/integration/add_to_rails_app_spec.rb index 5755b07..8e734eb 100644 --- a/spec/integration/add_to_rails_app_spec.rb +++ b/spec/integration/add_to_rails_app_spec.rb @@ -109,35 +109,6 @@ def run(command) expect(include_line).to_not be_nil,lines.inspect end - it "inserts can update old configuration" do - run "bin/rails generate stitches:api" - - initializer = rails_root / "config" / "initializers" / "stitches.rb" - - initializer_contents = File.read(initializer).split(/\n/) - found_initializer = false - File.open(initializer,"w") do |file| - initializer_contents.each do |line| - if line =~ /allowlist/ - line = line.gsub("allowlist","whitelist") - found_initializer = true - end - file.puts line - end - end - - raise "Didn't find 'allowlist' in the initializer?!" if !found_initializer - - run "bin/rails generate stitches:update_configuration" - - lines = File.read(initializer).split(/\n/) - include_line = lines.detect { |line| - line =~ /whitelist/ - } - - expect(include_line).to be_nil,lines.inspect - end - class RoutesFileAnalysis attr_reader :routes_file def initialize(routes_file, namespace: nil, module_scope: nil, resource: nil, mounted_engine: nil) diff --git a/stitches.gemspec b/stitches.gemspec index 76afc76..ccdd164 100644 --- a/stitches.gemspec +++ b/stitches.gemspec @@ -20,6 +20,7 @@ Gem::Specification.new do |s| s.add_runtime_dependency("rails") s.add_runtime_dependency("pg") + s.add_runtime_dependency("lru_redux") s.add_development_dependency("rspec", ">= 3") s.add_development_dependency("rake")