Skip to content

Commit 294535a

Browse files
author
Jacob Cantwell
committed
0 parents  commit 294535a

File tree

366 files changed

+13268
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

366 files changed

+13268
-0
lines changed

.DS_Store

12 KB
Binary file not shown.

CODE_OF_CONDUCT.md

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# Code of conduct
2+
3+
This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
4+
5+
For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
6+
[email protected] with any additional questions or comments.

CONTRIBUTING.md

+61
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# Contributing Guidelines
2+
3+
Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional
4+
documentation, we greatly value feedback and contributions from our community.
5+
6+
Please read through this document before submitting any issues or pull requests to ensure we have all the necessary
7+
information to effectively respond to your bug report or contribution.
8+
9+
10+
## Reporting Bugs/Feature Requests
11+
12+
We welcome you to use the GitHub issue tracker to report bugs or suggest features.
13+
14+
When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already
15+
reported the issue. Please try to include as much information as you can. Details like these are incredibly useful:
16+
17+
* A reproducible test case or series of steps
18+
* The version of our code being used
19+
* Any modifications you've made relevant to the bug
20+
* Anything unusual about your environment or deployment
21+
22+
23+
## Contributing via Pull Requests
24+
Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that:
25+
26+
1. You are working against the latest source on the *master* branch.
27+
2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already.
28+
3. You open an issue to discuss any significant work - we would hate for your time to be wasted.
29+
30+
To send us a pull request, please:
31+
32+
1. Fork the repository.
33+
2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change.
34+
3. Ensure local tests pass.
35+
4. Commit to your fork using clear commit messages.
36+
5. Send us a pull request, answering any default questions in the pull request interface.
37+
6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.
38+
39+
GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and
40+
[creating a pull request](https://help.github.com/articles/creating-a-pull-request/).
41+
42+
43+
## Finding contributions to work on
44+
Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start.
45+
46+
47+
## Code of Conduct
48+
This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
49+
For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
50+
[email protected] with any additional questions or comments.
51+
52+
53+
## Security issue notifications
54+
If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.
55+
56+
57+
## Licensing
58+
59+
See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
60+
61+
We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes.

LICENSE

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
2+
3+
Permission is hereby granted, free of charge, to any person obtaining a copy of
4+
this software and associated documentation files (the "Software"), to deal in
5+
the Software without restriction, including without limitation the rights to
6+
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
7+
the Software, and to permit persons to whom the Software is furnished to do so.
8+
9+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
10+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
11+
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
12+
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
13+
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
14+
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
15+

README.md

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Read Analogue Gauges With Machine Learning on AWS
2+
3+
Analogue pointer-type dials are widely applied in many industries such as Oil & Gas. Visual inspection to remote oil well sites is necessary to obtain readings from analog gauges. This means a human operator must travel to the gauge’s location, read its current value, and log that value to enable the data to be used elsewhere. This is a time consuming and expensive process and it is prone to human errors. An alternative to the manual readings is to record a video of the gauge and analyze the pressure variations using computer vision techniques. This project shows how to use AWS IoT services to read analogue gauges remotely.
4+
5+
[Read Analogue Gauges With Machine Learning on AWS](https://aws-computer-vision.jacobcantwell.com/)

archetypes/default.md

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
title: "{{ replace .Name "-" " " | title }}"
3+
date: {{ .Date }}
4+
---
5+

config.toml

+69
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
baseURL = "/"
2+
languageCode = "en-US"
3+
defaultContentLanguage = "en"
4+
5+
title = "Read Analogue Gauges With Machine Learning on AWS"
6+
theme = "hugo-theme-learn"
7+
metaDataFormat = "yaml"
8+
defaultContentLanguageInSubdir= "true"
9+
uglyurls = "true"
10+
enableEmoji = true
11+
12+
[params]
13+
# Prefix URL to edit current page. Will display an "Edit this page" button on top right hand corner of every page.
14+
# Useful to give opportunity to people to create merge request for your doc.
15+
# See the config.toml file from this documentation site to have an example.
16+
editURL = "https://github.com/jacobcantwell/read-analogue-gauges-with-machine-learning-on-aws/blob/main/content/"
17+
# Author of the site, will be used in meta information
18+
author = "Jacob Cantwell"
19+
# Description of the site, will be used in meta information
20+
description = "Read Analogue Gauges With Computer Vision"
21+
# Shows a checkmark for visited pages on the menu
22+
showVisitedLinks = true
23+
# Disable search function. It will hide search bar
24+
disableSearch = false
25+
# Javascript and CSS cache are automatically busted when new version of site is generated.
26+
# Set this to true to disable this behavior (some proxies don't handle well this optimization)
27+
disableAssetsBusting = false
28+
# Set this to true to disable copy-to-clipboard button for inline code.
29+
disableInlineCopyToClipBoard = false
30+
# A title for shortcuts in menu is set by default. Set this to true to disable it.
31+
disableShortcutsTitle = false
32+
# When using mulitlingual website, disable the switch language button.
33+
disableLanguageSwitchingButton = true
34+
# Hide breadcrumbs in the header and only show the current page title
35+
disableBreadcrumb = false
36+
# Hide Next and Previous page buttons normally displayed full height beside content
37+
disableNextPrev = false
38+
# Order sections in menu by "weight" or "title". Default to "weight"
39+
ordersectionsby = "weight"
40+
# Change default color scheme with a variant one. Can be "red", "blue", "green".
41+
themeVariant = "mine"
42+
43+
[outputs]
44+
home = [ "HTML", "RSS", "JSON"]
45+
46+
[[menu.shortcuts]]
47+
name = "Analogue Gauge Builder"
48+
url = "gauge/?g=bp30,bp70,bp110,bp150,bp190,230"
49+
weight = 10
50+
51+
[[menu.shortcuts]]
52+
name = "AWS Workshops"
53+
url = "https://workshops.aws/"
54+
weight = 20
55+
56+
[[menu.shortcuts]]
57+
name = "GitHub - Read Analogue Gauges With Machine Learning on AWS"
58+
url = "https://github.com/jacobcantwell/read-analogue-gauges-with-machine-learning-on-aws"
59+
weight = 30
60+
61+
[[menu.shortcuts]]
62+
name = "Workshop notes"
63+
url = "notes.html"
64+
weight = 31
65+
66+
[markup]
67+
defaultMarkdownHandler = "blackfriday"
68+
[markup.blackFriday]
69+
hrefTargetBlank = true

content/.DS_Store

14 KB
Binary file not shown.

content/0_introduction/.DS_Store

8 KB
Binary file not shown.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
+++
2+
title = "Introduction"
3+
chapter = false
4+
weight = 10
5+
+++
6+
7+
Analogue pointer-type dials are widely applied in many industries such as Oil & Gas. Visual inspection to remote oil well sites is necessary to obtain readings from analog gauges. This means a human operator must travel to the gauge’s location, read its current value, and log that value to enable the data to be used elsewhere. This is a time consuming and expensive process and it is prone to human errors. An alternative to the manual readings is to record a video of the gauge and analyze the pressure variations using computer vision techniques. This project shows how to use AWS IoT services to read analogue gauges remotely.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
+++
2+
title = "Benefits"
3+
chapter = false
4+
weight = 20
5+
+++
6+
7+
We see teams across the industry turn to this practice to reduce incidents, lower the cost of downtime, train their team and prepare for moments that matter.
8+
9+
### Preventive maintenance
10+
11+
With machine learning you are able to detect issues before they become incidents and before your customers get affected.
12+
13+
### Cost of downtime
14+
15+
The cost of downtime is very expensive, the average company loses $300,000/hour of downtime. The numbers get larger with the companies that operate at internet scale, and if it's during a high traffic event like a sale or launch.
16+
17+
### Train your team
18+
19+
Give your team a chance to build their muscle memory. Run planned machine learning experiments to train your engineers.
20+
21+
### Prepare for moments that matter
22+
23+
By working on the three points above your team will feel more confident everyday as they build their respective applications and systems. Less time will be spent fire fighting and more time building and creating value.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
+++
2+
title = "Considerations"
3+
chapter = false
4+
weight = 70
5+
+++
6+
7+
To implement Machine Learning, one should follow the scientific method to implement experiments:
8+
9+
1. Observe Your System
10+
2. Baseline your metrics
11+
3. Form a Hypothesis with **Abort Conditions**
12+
4. Define **Blast Radius**
13+
5. Run Experiment
14+
6. Analyze Results
15+
7. Expand Scope and Re-Test
16+
8. Share Results
17+
18+
**Blast Radius** is the number of hosts, containers or resources that are targeted in an experiment. This is also known as the subset of a system that can be impacted by an attack; the worst case impact of a failed experiment.
19+
20+
For example, usually measured in customer impact (i.e. 10% of customers could be impacted), but may be expressed in hosts, services, or containers.
21+
22+
**Magnitude** is how the intensity of the attack you’re running is defined, and can also be defined as the impact that an experiment has.
23+
24+
For example, a CPU attack would have a different magnitude if it targeted 10% of CPU versus 20% of CPU.
25+
26+
**Abort Conditions** are the conditions that would cause to you to press the halt button. They are system conditions that indicate when we should stop a chaos experiment in order to avoid accidental damage
27+
28+
One should always define Abort Conditions. Examples of abort conditions include SLAs, Error Rates, Availability, Latency, Traffic, and any other KPI that matters to your organization.
29+
30+
When running chaos experiments, it is recommended to start with a small blast radius and magnitude. As you run more experiments and build more confidence, you can increase the blast radius and magnitude.
8 KB
Binary file not shown.
+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
+++
2+
title = "Part x: Analogue gauge"
3+
chapter = true
4+
weight = 20
5+
+++
6+
7+
#### Analogue gauge
8+
9+
How to create analogue gauge
10+
11+
{{% children showhidden="false" %}}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
+++
2+
title = "1.x Real gauge"
3+
chapter = false
4+
weight = 20
5+
+++
6+
7+
Discuss the build of the real gauge
8+
9+
* Physical build
10+
* Tips for training data
11+
12+
![Pressure gauge](10_real/images/IMG_0766.jpg)
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
+++
2+
title = "1.x Virtual gauge"
3+
chapter = false
4+
weight = 30
5+
+++
6+
7+
## Create a virtual gauge
8+
9+
![Pressure gauge](20_virtual/images/virtual-gauge-1.png)
10+
11+
[View a simulated pressure gauge](/gauge/?bp=100)
Binary file not shown.
Loading
Loading

content/0_introduction/_index.md

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
---
2+
title: "Introduction"
3+
weight: 5
4+
chapter: true
5+
draft: false
6+
---
7+
8+
## Learning Objectives
9+
10+
Today we are going to learn the following topics:
11+
12+
* A gauge
13+
* A camera
14+
* Cloud services
15+
* Centralized dashboard

content/10_prerequisites/10_cloud.md

+69
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
---
2+
title: "Update IAM settings for your Workspace"
3+
chapter: false
4+
weight: 19
5+
---
6+
7+
{{% notice info %}}
8+
Cloud9 normally manages IAM credentials dynamically. This isn't currently compatible with
9+
the EKS IAM authentication, so we will disable it and rely on the IAM role instead.
10+
{{% /notice %}}
11+
12+
- Return to your workspace and click the gear icon (in top right corner), or click to open a new tab and choose "Open Preferences"
13+
- Select **AWS SETTINGS**
14+
- Turn off **AWS managed temporary credentials**
15+
- Close the Preferences tab
16+
![c9disableiam](/images/c9disableiam.png)
17+
18+
To ensure temporary credentials aren't already in place we will also remove
19+
any existing credentials file:
20+
```sh
21+
rm -vf ${HOME}/.aws/credentials
22+
```
23+
Install jq - jq is a command-line tool for parsing JSON.
24+
```sh
25+
sudo yum install jq
26+
```
27+
We should configure our aws cli with our current region as default.
28+
29+
{{% notice info %}}
30+
If you are [at an AWS event](https://eksworkshop.com/020_prerequisites/aws_event/), ask your instructor which **AWS region** to use.
31+
{{% /notice %}}
32+
33+
```sh
34+
export ACCOUNT_ID=$(aws sts get-caller-identity --output text --query Account)
35+
export AWS_REGION=$(curl -s 169.254.169.254/latest/dynamic/instance-identity/document | jq -r '.region')
36+
```
37+
38+
Check if AWS_REGION is set to desired region
39+
```sh
40+
test -n "$AWS_REGION" && echo AWS_REGION is "$AWS_REGION" || echo AWS_REGION is not set
41+
```
42+
43+
Let's save these into bash_profile
44+
```sh
45+
echo "export ACCOUNT_ID=${ACCOUNT_ID}" | tee -a ~/.bash_profile
46+
echo "export AWS_REGION=${AWS_REGION}" | tee -a ~/.bash_profile
47+
aws configure set default.region ${AWS_REGION}
48+
aws configure get default.region
49+
```
50+
51+
### Validate the IAM role
52+
53+
Use the [GetCallerIdentity](https://docs.aws.amazon.com/cli/latest/reference/sts/get-caller-identity.html) CLI command to validate that the Cloud9 IDE is using the correct IAM role.
54+
55+
```
56+
aws sts get-caller-identity --query Arn | grep Pulumi-Workshop-Admin -q && echo "IAM role valid" || echo "IAM role NOT valid"
57+
```
58+
59+
<!--
60+
First, get the IAM role name from the AWS CLI.
61+
```bash
62+
INSTANCE_PROFILE_NAME=`basename $(aws ec2 describe-instances --filters Name=tag:Name,Values=aws-cloud9-${C9_PROJECT}-${C9_PID} | jq -r '.Reservations[0].Instances[0].IamInstanceProfile.Arn' | awk -F "/" "{print $2}")`
63+
aws iam get-instance-profile --instance-profile-name $INSTANCE_PROFILE_NAME --query "InstanceProfile.Roles[0].RoleName" --output text
64+
```
65+
-->
66+
67+
If the IAM role is not valid, <span style="color: red;">**DO NOT PROCEED**</span>. Go back and confirm the steps on this page.
68+
69+
If youv'e reached the end of this section you can skip to the [**Workshop Setup**](/15_workshop_setup/50_workshop_setup.html) section.

content/10_prerequisites/_index.md

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
+++
2+
title = "Prerequisites"
3+
chapter = true
4+
weight = 10
5+
+++
6+
7+
# Prerequisites
8+
9+
{{% children showhidden="false" %}}
10+
11+

0 commit comments

Comments
 (0)