Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhanced Storage Classes #19

Open
wants to merge 13 commits into
base: master
Choose a base branch
from

Conversation

ambsw-technology
Copy link

@ambsw-technology ambsw-technology commented May 1, 2019

This PR is built on #18 (for easy merging) but is mostly unrelated. It introduces a variety of improvements to the storage engines (i.e. YAMLFile and ParamterStore) with related changes to the CLI experience:

  • YAMLFile accepts a root_path kwarg (default=/, env variable SSM_YAML_ROOT). The root of the YAML file will be equivalent to this path in the Parameter Store when reading and writing (closes Root Path Config #11).
  • ParameterStore accepts a no_secure kwarg (default=False, env variable SSM_NO_SECURE). If true, it will not request secure parameters from ParameterStore (closes Ignore Encrypted Entries #13)
  • ParameterStore coerces basic types (e.g. int, float, NoneType) so a YAML integer and SSM string are recognized as a match if they're otherwise equal.
  • ParameterStore now supports the StringList type (i.e. a list of strings) and is more aggressive about type checking.
  • The -p flag has been converted to the env variable SSM_PATHS to increase safety (resolves Replace -p with ENV Variable #15). The env variable accepts : and ; as separators (the kwarg expects a tuple/list/set).

However, the more significant improvement is the introduction of metadata in the YAML files. YAML files now store the root_path, paths and no_secure settings used to generate them. If a subsequent command attempts to use the file with incompatible configurations, it will throw an exception.

This is a more general solution to my concerns in #15. It can be extended to cover all relevant parameters, whether used through the CLI or programmatically. For example, it prevents the following sequence of events:

> set SSM_PATHS=/my/deep/path
> ssh-diff init
# close the command prompt and reopen, implicitly resetting the PATHS env variable:
> set SSM_PATHS=/
# anything outside /my/deep/path would be deleted
> ssh-diff apply
ValueError: Path / was not included in this file when it was created.

However, it's smart enough to permit subsets of the original scope:

> set PATHS=/
> ssh-diff init
# because this path is contained in our original scope, it's "safe" to update that branch
> set PATHS=/test
# change /other/key and /test/child/key
# PATHS continues to limit updates to the relevant branch 
> ssh-diff plan
~/test/child/key:
        < original
        > changed

P.S. This PR may contain small fixes to previous PRs.

EDIT: Encapsulated the type checking and coercion in ParameterStore. I had originally implemented it in the DiffResolver, but realized that ParameterStore is really the limiting factor here. In theory, you could use the DiffResolver with a less restrictive endpoint (Azure? Vault? AWS Secrets?) and might want different coercion and type checking.

…, and better encapsulate behaviors (among other things to simplify testing)
…lified/standardized the interface (i.e. `configure`, `merge`, `plan`, and `__init__`)
@claytondaley claytondaley force-pushed the storage-improvements branch 2 times, most recently from 8968375 to 7614275 Compare May 1, 2019 22:12
…nd line flags to ENV variables (fixes runtheops#15), (2) a way to generate YAML files for branches of the SSM tree (closes runtheops#11), (3) the ability to ignore SecureString keys if they are not necessary (closes runtheops#13), (4) support for the SSM StringList type and more timely type coercion so e.g. YAML integers and SSM strings match, and (5) the introduction of metadata in the YAML files to permit compatibility checking (more general fix for runtheops#15 with support for new features)
@claytondaley claytondaley force-pushed the storage-improvements branch from 7614275 to d01c7dc Compare May 1, 2019 22:57
…` class to simplify inserts (both backwards-compatible to the `SecureTag`. For safer backups, an `AWS_NO_DECRYPT` (i.e. kwarg `no_decrypt`) was added. README was updated and the CLI commands were fully converted to `git` style names.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Replace -p with ENV Variable Ignore Encrypted Entries Root Path Config
2 participants