Skip to content

Commit

Permalink
Remove ppt presenter (#27)
Browse files Browse the repository at this point in the history
  • Loading branch information
pcstout authored Nov 18, 2021
1 parent 9206565 commit 8c726a9
Show file tree
Hide file tree
Showing 24 changed files with 15,144 additions and 742 deletions.
3 changes: 1 addition & 2 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
language: python
python:
- "3.6"
- "3.8"
before_install:
- export LOG_LEVEL=DEBUG
- export BOTO_CONFIG=/dev/null
Expand All @@ -11,4 +11,3 @@ script:
- pytest --cov -v
after_success:
- coveralls

39 changes: 35 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,22 @@ test:
pytest -v --cov --cov-report=term --cov-report=html


.PHONY: deploy_development
deploy_development:
sls deploy --stage development


.PHONY: deploy_dev
deploy_dev:
sls deploy --stage dev
deploy_dev: deploy_development


.PHONY: remove_development
remove_development:
sls remove --stage development


.PHONY: remove_dev
remove_dev:
sls remove --stage dev
remove_dev: remove_development


.PHONY: deploy_staging
Expand All @@ -48,3 +56,26 @@ deploy_production:
.PHONY: remove_production
remove_production:
sls remove --stage production


.PHONY: man_test_all
man_test_all:
JSON=$$(./scripts/json_to_gql.py tests/handlers/test_json/create_syn_project.json name="Manual Test Project `date +%s`" | sls invoke -f graphql --stage development); \
echo "$$JSON"; \
ID=$$(echo $$JSON | python -c "import json,sys;obj=json.loads(' '.join(sys.stdin.read().split(')', maxsplit=1)[1:]));print(json.loads(obj['body'])['data']['createSynProject']['synProject']['id']);"); \
./scripts/json_to_gql.py tests/handlers/test_json/update_syn_project.json id=$$ID name="Manual Test Project UPDATED `date +%s`" | sls invoke -f graphql --stage development; \
./scripts/json_to_gql.py tests/handlers/test_json/get_syn_project.json id=$$ID | sls invoke -f graphql --stage development; \
./scripts/json_to_gql.py tests/handlers/test_json/create_slide_deck.json | sls invoke -f graphql --stage development;


.PHONY: man_test_create_syn_project
man_test_create_syn_project:
./scripts/json_to_gql.py tests/handlers/test_json/create_syn_project.json name="Manual Test Project `date +%s`" | sls invoke -f graphql --stage development


.PHONY: man_test_create_slide_deck
man_test_create_slide_deck:
./scripts/json_to_gql.py tests/handlers/test_json/create_slide_deck.json | sls invoke -f graphql --stage development



91 changes: 56 additions & 35 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,54 +5,63 @@

## Overview

A [Serverless](https://serverless.com/framework/docs/getting-started) application that runs on [AWS Lambda](https://aws.amazon.com/lambda) exposing a [GraphQL](https://graphql.org) interface for maintaining Sprints in [Synapse](https://www.synapse.org).
A [Serverless](https://serverless.com/framework/docs/getting-started) application that runs
on [AWS Lambda](https://aws.amazon.com/lambda) exposing a [GraphQL](https://graphql.org) interface for maintaining
Sprints in [Synapse](https://www.synapse.org).

Capabilities:

- Project:
- [Create](tests/handlers/test_json/create_syn_project.json)
- [Update](tests/handlers/test_json/update_syn_project.json)
- [Query](tests/handlers/test_json/get_syn_project.json)
- [Create](tests/handlers/test_json/create_syn_project.json)
- [Update](tests/handlers/test_json/update_syn_project.json)
- [Query](tests/handlers/test_json/get_syn_project.json)
- Slide Deck:
- [Create](tests/handlers/test_json/create_slide_deck.json)
- [Create](tests/handlers/test_json/create_slide_deck.json)

## Development Setup

- [Install the Serverless Framework](https://serverless.com/framework/docs/providers/aws/guide/quick-start)
- `npm install -g serverless`
- Configure your AWS credentials by following [these directions](https://serverless.com/framework/docs/providers/aws/guide/credentials)
- `npm install -g serverless`
- Configure your AWS credentials by
following [these directions](https://serverless.com/framework/docs/providers/aws/guide/credentials)
- Install Serverless Plugins:
- `npm install`
- `npm install`
- Create and activate a Virtual Environment:
- `python3 -m venv .venv`
- `source .venv/bin/activate`
- `python3.8 -m venv .venv`
- `source .venv/bin/activate`
- `python -m pip install --upgrade pip` (REQUIRED!)
- Configure environment variables:
- Copy each file in [templates](templates) into the project's root directory and edit each file to contain the correct values.
- Copy each file in [templates](templates) into the project's root directory and edit each file to contain the
correct values.
- Install Python Dependencies:
- `make reqs`
- `make reqs`
- Run tests.
- `make test`
- `make test`

## Deploying

- Populate SSM with the environment variables. This only needs to be done once or when the files/values change.
- `./scripts/set_ssm.py --stage <service-stage>`
- Example: `./scripts/set_ssm.py --stage production`
- See the [Authentication](#authentication) section for generating secrets and API keys.
- `./scripts/set_ssm.py --stage <service-stage> --action <import | delete>`
- Example: `./scripts/set_ssm.py --stage production --action import`
- See the [Authentication](#authentication) section for generating secrets and API keys.
- Create the `A` records in Route53 if using a custom domain. This only needs to be done once for each stage.
- `sls create_domain --stage <stage>`
- Example: - `sls create_domain --stage production`
- See [serverless-domain-manager](https://github.com/amplify-education/serverless-domain-manager) for more details on configuring your custom domain.
- `sls create_domain --stage <stage>`
- Example: - `sls create_domain --stage production`
- See [serverless-domain-manager](https://github.com/amplify-education/serverless-domain-manager) for more details
on configuring your custom domain.
- Deploy to AWS
- Deploy to "dev": `make deploy_dev`
- Deploy to "staging": `make deploy_staging`
- Deploy to "production": `make deploy_production`
- Deploy to "development": `make deploy_dev`
- Deploy to "staging": `make deploy_staging`
- Deploy to "production": `make deploy_production`

## Authentication

Authentication will be done using [API Gateway Lambda Authorizers](https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-use-lambda-authorizer.html).
Authentication will be done
using [API Gateway Lambda Authorizers](https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-use-lambda-authorizer.html)
.

Initially a simple JWT authentication mechanism will be used to secure this service. A more robust authentication system will be implemented at a later date.
Initially a simple JWT authentication mechanism will be used to secure this service. A more robust authentication system
will be implemented at a later date.

A secret will be stored in an environment variable along with a comma separated list of API keys.

Expand All @@ -64,18 +73,30 @@ JWT_API_KEYS=key1,key2,key3
The process for allowing a client access to the service is as follows:

1. Generate a secret key and an API key by running [gen_key.py](scripts/gen_key.py).
- Add the keys to your `private.ssm.env.json` file.
- Update SSM: `./scripts/set_ssm.py --stage <service-stage>`
2. Generate a JWT for the client by running [gen_jwt.py](scripts/gen_jwt.py). Use the secret and API key generated above.
- Add the keys to your `private.ssm.env.json` file.
- Update SSM: `./scripts/set_ssm.py --stage <service-stage> --action import`
2. Generate a JWT for the client by running [gen_jwt.py](scripts/gen_jwt.py). Use the secret and API key generated, or a
stage to load from the configuration file.

## Manual Testing

- View Logs: `sls logs -f graphql --tail`
- Test Queries:
- Project:
- Create: `./scripts/json_to_gql.py tests/handlers/test_json/create_syn_project.json | sls invoke -f graphql --stage dev`
- Update: `./scripts/json_to_gql.py tests/handlers/test_json/update_syn_project.json | sls invoke -f graphql --stage dev`
- Query: `./scripts/json_to_gql.py tests/handlers/test_json/get_syn_project.json | sls invoke -f graphql --stage dev`
- Slide Deck:
- Create: `./scripts/json_to_gql.py tests/handlers/test_json/create_slide_deck.json | sls invoke -f graphql --stage dev`
- With curl: `curl -X POST -H 'Authorization: Bearer JWT_TOKEN_HERE' --data 'QUERY_HERE' ENDPOINT_URL_HERE`
- Project:
-
Create: `./scripts/json_to_gql.py tests/handlers/test_json/create_syn_project.json | sls invoke -f graphql --stage development`
-
Update: `./scripts/json_to_gql.py tests/handlers/test_json/update_syn_project.json | sls invoke -f graphql --stage development`
-
Query: `./scripts/json_to_gql.py tests/handlers/test_json/get_syn_project.json | sls invoke -f graphql --stage development`
-
- Slide Deck:
-
Create: `./scripts/json_to_gql.py tests/handlers/test_json/create_slide_deck.json | sls invoke -f graphql --stage development`
-
- Test all four queries with: `make man_test_all`


- With curl: `curl -X POST -H 'Authorization: Bearer JWT_TOKEN_HERE' --data 'QUERY_HERE' ENDPOINT_URL_HERE/graphql`
-
Example: `curl -X POST -H 'Authorization: Bearer abcDEF.GhIJv4' --data '{"query": "query GetSynProject($id: String!) { synProject(id: $id) { id name } }","variables": {"id": "syn123456789"}}' https://api.my-domain.com/graphql`
1 change: 1 addition & 0 deletions core/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
from .auth import Auth
from .env import Env
from .synapse import Synapse
from .config import Config
2 changes: 1 addition & 1 deletion core/auth.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,4 +25,4 @@ def decode_jwt(cls, token, secret):

@classmethod
def encode_jwt(cls, secret, api_key):
return jwt.encode({'apiKey': api_key}, secret, algorithm='HS256').decode()
return jwt.encode({'apiKey': api_key}, secret, algorithm='HS256')
96 changes: 96 additions & 0 deletions core/config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
import os
import json
import yaml


class Config:
class Stages:
PRODUCTION = 'production'
STAGING = 'staging'
DEVELOPMENT = 'development'
TEST = 'test'
ALL = [PRODUCTION, STAGING, DEVELOPMENT, TEST]

@classmethod
def load_local_into_env(cls, filename, stage=None):
"""Loads and sets OS environment variables from a local config file.
This should only be called when locally developing, running tests, or setting remote SSM key/values.
In production or CI all variables must be set in the OS environment or SSM and not loaded from a file.
Args:
filename: The filename of the config file to open.
stage: Which stage to load from the config file (if JSON config file).
Returns:
Dictionary of key/values.
"""
env_vars = cls.open_local(filename, stage=stage, for_env=True)

for key, value in env_vars.items():
if value is None:
print('Environment variable: {0} has no value and will not be set.'.format(key))
else:
if isinstance(value, bool):
value = str(value).lower()
elif not isinstance(value, str):
value = str(value)
os.environ[key] = value

return env_vars

@classmethod
def open_local(cls, filename, stage=None, for_env=False):
"""Opens a local config file and parses it.
Args:
filename: The filename of the config file to open.
stage: Which stage to load from the config file (if JSON config file).
for_env: Set to True if loading a file to set ENV variables.
Returns:
Dict of environment variables.
"""
module_dir = os.path.dirname(os.path.abspath(__file__))
src_root_dir = os.path.abspath(os.path.join(module_dir, '..'))
config_file_path = os.path.join(src_root_dir, filename)

result = {}

if os.path.isfile(config_file_path):
print('Loading local configuration from: {0}'.format(config_file_path))

if filename.endswith('.yml'):
config = yaml.safe_load(cls._read_file(config_file_path))
else:
config = json.loads(cls._read_file(config_file_path)).get(stage)

for key, value in config.items():
parsed_value = value

if str(value).startswith('$ref:'):
filename = value.replace('$ref:', '')
parsed_value = cls._read_file(filename)

if for_env:
if isinstance(parsed_value, list):
if len(parsed_value) > 0:
if isinstance(parsed_value[0], dict):
parsed_value = json.dumps(parsed_value)
else:
parsed_value = ','.join(parsed_value)
else:
parsed_value = ''
elif isinstance(parsed_value, dict):
parsed_value = json.dumps(parsed_value)

result[key] = parsed_value
else:
print('Configuration file not found at: {0}'.format(config_file_path))

return result

@classmethod
def _read_file(cls, path):
with open(path, mode='r') as f:
return f.read()
36 changes: 18 additions & 18 deletions core/env.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,41 +2,41 @@


class Env:
@classmethod
def SERVICE_NAME(cls, default=None):
@staticmethod
def SERVICE_NAME(default=None):
"""
This variable must be set on the OS (not on SSM)
"""
return ParamStore.get('SERVICE_NAME', default=default, store=ParamStore.Stores.OS).value

@classmethod
def SERVICE_STAGE(cls, default=None):
@staticmethod
def SERVICE_STAGE(default=None):
"""
This variable must be set on the OS (not on SSM)
"""
return ParamStore.get('SERVICE_STAGE', default=default, store=ParamStore.Stores.OS).value

@classmethod
def LOG_LEVEL(cls, default=None):
@staticmethod
def LOG_LEVEL(default=None):
return ParamStore.get('LOG_LEVEL', default=default).value

@classmethod
def SYNAPSE_USERNAME(cls, default=None):
@staticmethod
def SYNAPSE_USERNAME(default=None):
return ParamStore.get('SYNAPSE_USERNAME', default=default).value

@classmethod
def SYNAPSE_PASSWORD(cls, default=None):
@staticmethod
def SYNAPSE_PASSWORD(default=None):
return ParamStore.get('SYNAPSE_PASSWORD', default=default).value

@classmethod
def JWT_SECRET(cls, default=None):
@staticmethod
def JWT_SECRET(default=None):
"""
Secret key used to encode JWTs.
"""
return ParamStore.get('JWT_SECRET', default=default).value

@classmethod
def JWT_API_KEYS(cls, default=None):
@staticmethod
def JWT_API_KEYS(default=None):
"""
String of comma separated keys that are used for API access.
Expand All @@ -45,10 +45,10 @@ def JWT_API_KEYS(cls, default=None):
"""
return ParamStore.get('JWT_API_KEYS', default=default).to_list(delimiter=',')

@classmethod
def SLIDE_DECKS_BUCKET_NAME(cls, default=None):
@staticmethod
def SLIDE_DECKS_BUCKET_NAME(default=None):
return ParamStore.get('SLIDE_DECKS_BUCKET_NAME', default=default).value

@classmethod
def SLIDE_DECKS_URL_EXPIRES_IN_SECONDS(cls, default=300):
@staticmethod
def SLIDE_DECKS_URL_EXPIRES_IN_SECONDS(default=300):
return ParamStore.get('SLIDE_DECKS_URL_EXPIRES_IN_SECONDS', default=default).to_int()
4 changes: 2 additions & 2 deletions core/synapse.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,10 +49,10 @@ def client(cls):
"""
if not cls._synapse_client:
# Lambda can only write to /tmp so update the CACHE_ROOT_DIR.
synapseclient.cache.CACHE_ROOT_DIR = os.path.join(tempfile.gettempdir(), 'synapseCache')
synapseclient.core.cache.CACHE_ROOT_DIR = os.path.join(tempfile.gettempdir(), 'synapseCache')

# Multiprocessing is not supported on Lambda.
synapseclient.config.single_threaded = True
synapseclient.core.config.single_threaded = True

syn_user = Env.SYNAPSE_USERNAME()
syn_pass = Env.SYNAPSE_PASSWORD()
Expand Down
Loading

0 comments on commit 8c726a9

Please sign in to comment.