Amida Messaging Microservice

Design

API Spec

The spec can be viewed at https://amida-tech.github.io/amida-messaging-microservice/.

To update the spec, first edit the files in the docs directory. Then run aglio -i docs/src/docs.md --theme flatly -o index.html.

Merge the resulting changes to the gh-pages branch of the repository.

Features

Feature	Summary
ES6 via Babel	ES6 support using Babel.
Code Linting	Linting with eslint
Auto server restart	Restart the server using nodemon in real-time anytime an edit is made, with babel compilation and eslint.
ES6 Code Coverage via istanbul	Supports code coverage of ES6 code using istanbul and mocha. Code coverage reports are saved in coverage/ directory post yarn test execution. Open coverage/lcov-report/index.html to view coverage report. yarn test also displays code coverage summary on console. Code coverage can also be enforced overall and per file as well, configured via .istanbul.yml
Debugging via debug	Instead of inserting and deleting console.log you can replace it with the debug function and just leave it there. You can then selectively debug portions of your code by setting DEBUG env variable. If DEBUG env variable is not set, nothing is displayed to the console.
Promisified Code via bluebird	We love promise, don't we ? All our code is promisified and even so our tests via supertest-as-promised.
API parameter validation via express-validation	Validate body, params, query, headers and cookies of a request (via middleware) and return a response with errors; if any of the configured validation rules fail. You won't anymore need to make your route handler dirty with such validations.
Pre-commit hooks	Runs lint and tests before any commit is made locally, making sure that only tested and quality code is committed
Secure app via helmet	Helmet helps secure Express apps by setting various HTTP headers.
Uses yarn over npm	Uses new released yarn package manager by facebook. You can read more about it here

• CORS support via cors
• Uses http-status to set http status code. It is recommended to use httpStatus.INTERNAL_SERVER_ERROR instead of directly using 500 when setting status code.
• Has .editorconfig which helps developers define and maintain consistent coding styles between different editors and IDEs.

Developing locally

Install yarn:

npm install -g yarn

Install dependencies:

yarn

Set environment vars:

cp .env.example .env

Start server:

# Start server
yarn start

# Selectively set DEBUG env var to get logs
DEBUG=amida-messaging-microservice:* yarn start

Tests:

Create a JWT with the username value 'user0' and set TEST_TOKEN={token} in your .env file or an evironment variable. You can easily create a token using the amida-auth-microservice

# Run tests written in ES6
yarn test

# Run test along with code coverage
yarn test:coverage

# Run tests on file change
yarn test:watch

# Run tests enforcing code coverage (configured via .istanbul.yml)
yarn test:check-coverage

Lint:

# Lint code with ESLint
yarn lint

# Run lint on any file change
yarn lint:watch

Other gulp tasks:

# Wipe out dist and coverage directory
gulp clean

# Default task: Wipes out dist and coverage directory. Compiles using babel.
gulp

Deployment

Manual deployment with pm2

# compile to ES5
1. yarn build

# upload dist/ to your server
2. scp -rp dist/ user@dest:/path

# install production dependencies only
3. yarn --production

# Use any process manager to start your services
4. pm2 start dist/index.js

Deployment to AWS with Packer and Terraform

You will need to install pakcer and terraform installed on your local machine. Be sure to have your postgres host running and replace the pg_host value in the command below with the postgres host address.

1. First validate the AMI with a command similar to packer validate -var 'aws_access_key=myAWSAcessKey' -var 'aws_secret_key=myAWSSecretKey' -var 'build_env=development' -var 'logstash_host=logstash.amida.com' -var 'service_name=amida_messaging_microservice' -var 'ami_name=api-messaging-service-boilerplate' -var 'node_env=development' -var 'jwt_secret=My-JWT-Token' -var 'pg_host=amid-messages-packer-test.some_rand_string.us-west-2.rds.amazonaws.com' -var 'pg_db=amida_messages' -var 'pg_user=amida_messages' -var 'pg_passwd=amida-messages' template.json
2. If the validation from 1. above succeeds, build the image by running the same command but replacing validate with build
3. In the AWS console you can test the build before deployment. To do this, launch an EC2 instance with the built image and visit the health-check endpoint at <host_address>:4000/api/health-check. Be sure to launch the instance with security groups that allow http access on the app port (currently 4000) and access from Postgres port of the data base. You should see an "OK" response.
4. Enter aws_access_key and aws_secret_key values in the vars.tf file
5. run terraform plan to validate config
6. run terraform apply to deploy
7. To get SNS Alarm notifications be sure that you are subscribed to SNS topic arn:aws:sns:us-west-2:844297601570:ops_team_alerts and you have confirmed subscription

Further details can be found in the deploy directory.

Docker deployment

Docker Compose:

docker-compose up

Kubernetes Deployment

See the paper write-up for instructions on how to deploy with Kubernetes. The kubernetes.yml file contains the deployment definition for the project.

Logging

Universal logging library winston is used for logging. It has support for multiple transports. A transport is essentially a storage device for your logs. Each instance of a winston logger can have multiple transports configured at different levels. For example, one may want error logs to be stored in a persistent remote location (like a database), but all logs output to the console or a local file. We just log to the console for simplicity, but you can configure more transports as per your requirement.