#168 add initial devops guidelines

docs/devops-guidelines/.order

@@ -0,0 +1,8 @@
+DevOps-Guidelines
+Plan
+Create
+Verify
+Deploy
+Configuration-Management
+Monitoring
+Package

docs/devops-guidelines/Configuration-Management.md

@@ -0,0 +1,39 @@
+[DevOps Strategy](/Project-Artifacts/DevOps-Strategy)
+
+**Capabilities Inventory**
+
+|Capability|Ideal State|Tools In Inventory|
+|--|--|--|
+|Infrastructure Provisioning|Infrastructure is provisioned and configured as code along with the rest of the workload in an automated CI/CD pipeline|AWS CloudFormation, Azure Resource Manager|
+|Infrastructure Configuration|Infrastructure is provisioned and configured as code along with the rest of the workload in an automated CI/CD pipeline|AWS Config, AWS CLI, Azure Policy, Azure CLI|
+|Application Provisioning|Automatically create application CI in ServiceNow based on what gets deployed; users and administrators provisioned with proper access|ServiceNow, AzureAD, Okta|
+|Application Configuration|Configuration settings are exposed based on user roles.  Changes to configuration items are audited and sent through and appropriate approval process.|AWS OpWorks|
+
+**Infrastructure Provisioning**
+- Infrastructure will be provisioned using AWS CloudFormation templates.  This will allow the team to standardize on a tool several team members already know that the company is licensed for, has support for from AWS and will allow us to take advantage of all AWS native capabilities.  DCX is not intended to be cloud agnostic as it must take advantage of vendor specific services, e.g. CloudFront.
+
+- All CloudFormation templates will be stored in Azure DevOps and deployed use the Azure DevOps deployment pipeline.
+
+- Infrastructure provisioning will follow the [resource naming guidelines](https://phillips66.sharepoint.com/sites/IT_CloudRunway/DOH/Pages/Resource%20Naming.aspx) and [tagging guidelines](https://phillips66.sharepoint.com/sites/IT_CloudRunway/DOH/Pages/Resource%20Tags.aspx) outlined by CloudRunway.
+
+**Infrastructure Configuration**
+- Since all resources will be deployed and configured via code (Terraform, CloudFormation, etc.), configurations will be stored in version control (Git Repo).
+
+- Any updates will be reviewed via a Pull Request, then can be deployed by the service of choice. Terraform and CloudFormation both have the capability to update existing resources if possible, and if not, they'll replace any resources that cannot be updated in place.
+
+**Application Provisioning**
+- Applications will consist primarily of cloud services and AWS Lambda functions and will be designed to be serverless.
+
+- They will be provisioned using the [AWS Serverless Application Model (AWS SAM)](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/what-is-sam.html) templates.
+
+- The templates will be stored in Azure DevOps and deployed using the Azure DevOps deployment pipeline.
+
+**Application Configuration**
+- Application configuration information will be stored in different places depending on the vendor.
+
+- AWS Lambda functions will be configured according to the [AWS guidelines](https://docs.aws.amazon.com/lambda/latest/dg/lambda-configuration.html).
+
+- Sensitive information, e.g., passwords or API keys, will be stored in [AWS Secrets Manager](https://aws.amazon.com/secrets-manager/) based on guidelines to be provided by CloudRunway.
+
+- MuleSoft configuration will be stored and managed according to the MuleSoft best practices.  For more information on MuleSoft, go to the [MuleSoft site on Teams](https://teams.microsoft.com/l/team/19%3a90fdb0298b62405187a92b6ef779571d%40thread.skype/conversations?groupId=c3720085-0e68-401d-b28f-cdbf3ca059db&tenantId=4febecd1-b635-4bb7-96fd-6688487e52f1).
+

docs/devops-guidelines/Create.md

@@ -0,0 +1,66 @@
+[[_TOC_]]
+
+Specific tools used to create the source and related artifacts will be determined by a project team.
+
+These guidelines are intended to be general guidance and will be applicable to most teams.
+
+# Managing Artifacts
+For technical and software projects, a variety of artifacts are created. All artifacts should be maintined in git based repos 
+
+- Design artifacts such as high level designs, information architecutre, wire frames and similar should be committed to source and associated with a User Story whenever possible
+- Software developers create code in a branch created from a User Story or Bug/Defect
+
+All artifacts 
+
+Branching Strategies
+Repository Structure
+
+
+**Capabilities Inventory**
+
+|Capability|Ideal State|Tools In Inventory|
+|--|--|--|
+|Design|Designs are driven by user stories, captured ahead of development and vetted against a Definition of Ready (DoR) for design acceptance.  Design documents incorporate all aspects of the design, to include at a minimum UX/UI, infrastructure, data models, logic and integration. Design may optionally include a clickable prototype or a proof of concept.|Adobe XD, Zeplin, Adobe Photoshop, Adobe Illustrator|
+|Implementation|Developers work on tasks tied to user stories.  Unit tests are developed for acceptance criteria defined in the design ahead of development of the actual product.  Tasks are completed in priority order.  Development is accelerated by the use of IDEs and feedback tools.|Visual Studio, Visual Studio Code, MuleSoft, SQL Server Management Studio, Oracle SQL Developer, PL/SQL Developer, Automation Anywhere, PgAdmin, Resharper
+|Version Control|All artifacts are versioned and revisions to those artifacts are tracked by a version control system.  Merges are simple and reversion to a "known-good state" is easy.  The version control system supports distributed development and forms the backbone for the build/deployment pipelines.|git based Azure DevOps Repos |
+
+# Design
+Designs may consist of wireframes, mockups or prototypes. A wireframe is a low-fidelity visual representation of the site and will be used to depict the basic UI, detailing the layout and structure of the site. A mockup is a mid to high-fidelity visual representation of the site and will be used to highlight color scheme, visual style and typography. A prototype is a high-fidelity visual representation of the site and will be used to simulate user interaction by allowing the user to experience the content through clickable elements and interactions. In the event the design process needs to be accelerated, wireframes may be substituted with mockups.
+
+Designs will utilize Material Design, Font Awesome Icons and custom components. Phillips 66 brand guidelines will influence design concept and style. 
+
+Designs will be reviewed by the design team, development team and business analysts before handing off for development. User stories, acceptance criteria and UX personas are used to approve and validate designs. Optionally, user testing may be included in design review. 
+
+# Implementation
+Implementation is where the rubber meets the road.  The plans are laid.  The user stories are written.  The test cases have been identified and it's finally time to build something.
+
+Implementation will follow the SCRUM methodology and utilize Test Driven Development (TDD).  All features will be logged as user stories with acceptance criteria before they are worked.
+
+All development is expected to be performed using Visual Studio Code, Mulesoft Anypoint Studio and Eclipse.
+
+## Source Management
+
+Git repos in Azure DevOps are preferred. 
+
+Associate check ins using the #<work item number> in comments with Work Items to automatically generate release information as well as generate an audit trail
+
+All code is to be checked into a version control system on a regular basis.  More frequent check-ins provide benefits:
+- Team members can synchornize changes reducing efforts to code merge and staying up to date on changes
+- Continous integration builds can be triggered to verify code
+
+[Learning git](https://docs.microsoft.com/en-us/azure/devops/learn/git/what-is-git)
+
+
+## Coding Standards
+Developers will follow the appropriate coding standards for the languages being used.  Coding standards can be found at the following locations:
+
+* [Phillips 66 C# Coding Standard](https://phillips66.sharepoint.com/sites/IT_AE66PMO/Digital%20Capability/Shared%20Documents/2.2%20-%20IN%20-%20Business%20Improvement/WS%20-%20DCX/Working%20Documents/10.%20Development%20Standards/Phillips%2066%20C%20Sharp%20Coding%20Standard.docx?d=wde85208bab2c413e9412f255d55f1ce6&csf=1&e=LaONdH)
+* [Phillips 66 JavaScript Coding Standard](https://phillips66.sharepoint.com/sites/IT_AE66PMO/Digital%20Capability/Shared%20Documents/2.2%20-%20IN%20-%20Business%20Improvement/WS%20-%20DCX/Working%20Documents/10.%20Development%20Standards/Phillips%2066%20JavaScript%20Coding%20Standard.docx?d=w13ff595b078a47bca4db00eed07d381f&csf=1&e=SzX6fp)
+* [Phillips 66 SQL Coding Standard](https://phillips66.sharepoint.com/sites/IT_AE66PMO/Digital%20Capability/Shared%20Documents/2.2%20-%20IN%20-%20Business%20Improvement/WS%20-%20DCX/Working%20Documents/10.%20Development%20Standards/Phillips%2066%20SQL%20Coding%20Standard.docx?d=wf8d0654de3f1478ea6bb410a5f3f4515&csf=1&e=tfbVRr)
+
+## Unit Tests
+Unit tests will be developed for each user story that comply with the [Phillips 66 Automated Unit Testing Guidelines](https://phillips66.sharepoint.com/sites/IT_AE66PMO/Digital%20Capability/Shared%20Documents/2.2%20-%20IN%20-%20Business%20Improvement/WS%20-%20DCX/Working%20Documents/10.%20Development%20Standards/Phillips%2066%20Automated%20Unit%20Testing%20Guidelines.docx?d=w0b8bf2bafdf942f99230863565fe36da&csf=1&e=aHc8nb) and are adequate to test the feature.
+
+All features will be developed in accordance with the DCX architectural guidelines.  Specifically, the [Application Stacks (Development Framework)](/Project-Artifacts/Architectural-Overview/Application-Stacks-\(Development-Framework\)) guidelines and the [Backup and DR Strategy](/Project-Artifacts/Architectural-Overview/Backup-and-DR-Strategy).
+
+

docs/devops-guidelines/Create/Repository-Structure.md

@@ -0,0 +1,104 @@
+---
+
+# Introduction
+
+One of the bigger challenges to building software is managing the lifecycle of the source code, from authorship to deployment. Below is an outline of how to structure our source code, the goal of which is to achieve the following:
+
+- Any change must be easily reviewable, including how the change affects dependencies.
+- Building and deploying should be simple, fast, and reproducible.
+- Changes should be testable in both isolation and integrated with the existing environment.
+- Setting up new users/systems should require as few steps as possible and require little outside assistance.
+
+---
+
+# Review
+
+There are generally two approaches when setting up source control for a project:
+
+- Each application's code remains in its own repository
+- All code is in one repository (monorepo)
+
+Neither implementation is objectively better than the other; they are both highly dependent on the complexity of the application and delivery model. Below is a review of the benefits and challenges that come with each approach. 
+
+
+## <u>Separate Repositories</u>
+
+### Benefits
+
+- Organization  
+The structure of the repo is simple; it contains the source files and configuration from only one project.
+- Isolation  
+Each project exists in isolation. Changes can be made, and subsequently reviewed, knowing they have no direct or immediate impact on anything else.
+
+### Challenges
+
+- Dependencies
+Coordinating changes between multiple, dependent projects is a tedious process. It is made worse over a project's lifecycle as it naturally grows in complexity. One example of this would be a web app that is dependent on a component library. When the app requires a change to a component, the dependent component repo must go through its code review and release process before the web app is able to move forward. If there are further changes required of the updated component library, the cycle starts over and the delay increases.
+- Deployments
+The ability to automate build and release pipelines suffers when the dependency graph grows in complexity. Coordinating deployments requires a significant effort from the team to track interdependent versions and where they should exist in the release cycle.
+
+
+## <u>Single Repository (monorepo)</u>
+
+### Benefits
+
+- Organization
+All files exist in one, well-structured directory. Users can easily search across projects without having to sync updates and branches across multiple repositories.
+- Dependencies
+Every project moves forward together. Changes in dependencies should never block progress since review cycles and upstream integration never happen orthogonally.
+- Tooling
+Most applications and services operate on the file system. Providing immediate access to all required files greatly improves the speed, efficiency, and resiliency of the CI/CD pipelines.
+
+### Challenges
+
+- Scalability
+The life of a repository is long-lived. The VCS index continues to grow with every new file or changed line. While there are ways to mitigate this, the size of the repo will inevitably grow.
+
+---
+
+# Model Repository Implementation
+
+Our project is comprised of multiple slices of related functionality and will be developed in parallel by a distributed team. The current approach will maintain a single repository ("monorepo") containing top-level directories for each independent project.
+
+The structure of the repository will be similar to the following example:
+
+- <u>**Root**</u> (repo directory)
+  - Repo-level config files: e.g. `.gitignore`, `.gitattributes`, `.editorconfig`
+  - Makefile  (or similar) to provide top-level hooks into available scripts
+  - <u>**docs**</u>
+    - README.md
+    - installation.md
+  - <u>**scripts**</u> (utilities for cross-project coordination)
+    - README.md
+    - Makefile
+    - pipeline
+      - build.sh
+      - test.sh
+    - utils
+      - setup_machine.sh
+  - <u>**infrastructure**</u>
+    - README.md
+    - Makefile
+    - env
+      - ring1
+      - ring2
+    - modules
+      - lambda
+      - s3
+  - <u>**frontend**</u>
+    - Project-level config files: e.g. `.gitignore`, `.npmrc`, `setup.cfg`
+    - docs
+      - README.md
+      - installation.md
+    - scripts
+      - build.sh
+      - test.sh
+    - src
+    - tests
+  - <u>**api**</u>
+  - <u>**shared_components**</u>
+  - <u>**bloomreach**</u>
+
+
+---
+

docs/devops-guidelines/Deploy.md

@@ -0,0 +1,35 @@
+[[_TOC_]]
+
+Project code and artifacts are built locally using tools dictated by the software architecture of a project.
+
+Regardless of architecture, projects should automated builds to support *Continuous Integration* or *CI*, and, automated deployment to staged environments called [Rings](point to Rings defintion)
+
+# Continuous Integration
+
+# Continous Deployment
+
+# Ring Deployment
+
+*** insert favorite Ring picture here ***
+
+## Ring 1, Development
+Development testing, verifying services, proof of concepts and development spikes
+Complete access to the enviroment
+Restricted access to P66 systems
+
+## Ring 2, Integration Test
+Continues integration
+Deployed to automatically after successful builds
+Includes automated integration tests
+
+## Ring 3, Verify
+Deployed to on succesful builds in Ring 1
+
+## Ring 4, Stage
+Similar/same as Production
+May include a copy of Production data
+Used when manual verification by Stakeholders or Customers is required 
+
+## Ring 5, Launch
+Production environment
+Requires an approved ServiceNow CR prior to deployment

docs/devops-guidelines/DevOps-Guidelines.md

@@ -0,0 +1,28 @@
+# DevOps Approach
+
+"DevOps is cultural fusion of People, Process, and Technology to accelerate deliery of Business Value"
+
+![DevOps Continous Improvement](/.images/DevOps.png)
+
+A complete DevOps approach combines critial develop, operations and management concerns so that software projects can continously and consistently deliver high quality solutions. 
+
+Each practice within these guidelines works with others to form a working model for continous improvement. In addition, DevOps works hand in hand with portfolio and project management and agile initiatives.
+
+This wiki is intended for the practitioners: team members on projects. It  provides a set of guidelines that can be used as a starting point for any project's DevOps implementation. Each project will need to define specific tools and processes that will work best for that project. This wiki can be cloned and used as a starting point for a project specific DevOps Strategy.
+
+For an overview of DevOps, visit the Cloud Runway [DevOps Handbook](https://phillips66.sharepoint.com/sites/IT_DOH).
+
+Topics covered in this site include:
+- [Planning and managing work from inception to delivery](/Plan)
+- [Source management](/Create)
+- [Verifying quality](/Verify)
+- [Build and release automation](/Deploy)
+- [Monitoring a solution in production](/Monitor)
+
+This site assumes you are using and are familiar with [Azure DevOps](https://dev.azure.com).
+
+## Additional Information
+
+- DevOps Handbook
+- Patterns and Templates provided by Cloud Runway
+- learning resources

docs/devops-guidelines/Monitor.md

@@ -0,0 +1,33 @@
+[DevOps Strategy](/Project-Artifacts/DevOps-Strategy)
+
+**Capabilities Inventory**
+
+|Capability|Ideal State|Tools In Inventory|
+|--|--|--|
+|Performance Monitoring|Well established baselines and KPIs that can be tested against, monitored, and improved as a part of CI/CD pipeline and release cycles|AWS CloudWatch, Microsoft SCOM|
+|Security Monitoring|Automatically monitor and alert for appropriate credential rotation events, critical vulnerability patches, and user activity anomalies|AWS CloudTrail, SkyHigh, Dome9, QRadar|
+|Feedback Management|Build feedback submission tools into user interfaces to obtain feedback directly from customers; leverage Product Owner to translate feedback into feature-oriented user stories to be captured and prioritized by planning tools for future releases|Azure DevOps|
+|Consumption Management|Utilize CloudHealth to monitor and report off cloud costs across AWS & Azure; tags applied at resource creation based on ServiceNow application CI|CloudHealth|
+|Telemetry|Data is captured for all of the major types of telemetry data, business metrics, application metrics, infrastructure metrics and deployment pipeline metrics.  Telemetry data is analyzed and leveraged as KPIs to improve the product as part of the feedback loop.| |
+
+**Performance Monitoring**
+- DCX will follow the [guidance from CloudRunway](https://phillips66.sharepoint.com/sites/IT_CloudRunway/DOH/Pages/Monitoring%20and%20Logging.aspx) on monitoring and logging.
+
+- [Amazon CloudWatch Metrics](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-functions.html) will be used to monitor the performance of AWS Lambda functions against expectations.  
+
+- [Amazon CloudWatch Metrics](https://docs.aws.amazon.com/AmazonS3/latest/dev/cloudwatch-monitoring.html) will also be used to monitor the performance of AWS S3.
+
+- Other PaaS/SaaS vendors will be monitored using their native tools.  
+
+- BloomReach performance monitoring guidelines can be found [here](https://documentation.bloomreach.com/library/concepts/web-application/jmx-management-beans-support.html).
+
+**Security Monitoring**
+- DCX will follow the [guidance from CloudRunway](https://phillips66.sharepoint.com/sites/IT_CloudRunway/DOH/Pages/Security.aspx) on security best practices.
+
+**Feedback Management**
+- 
+**Consumption Management**
+- DCX will utilize CloudHealth to monitor and report cloud costs across AWS & Azure.  Tags will be applied in accordance with the guidance from Cloud Runway.
+
+**Telemetry**
+- Telemetry data will be captured [Amazon CloudWatch Metrics](https://docs.aws.amazon.com/AmazonS3/latest/dev/cloudwatch-monitoring.html) and recorded to [Azure Application Insights](https://docs.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview).