-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathtemplate.yaml
193 lines (183 loc) · 8.32 KB
/
template.yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
Monitoring Amazon S3 Backups with CloudWatch Dashboards and Alarms
https://github.com/aws-samples/monitor-backups-to-amazon-s3-using-amazon-cloudwatch
Parameters:
CreateBucket:
Description: Should this SAM template create a new Amazon S3 bucket, or does your bucket already exist? Please specify 'create' to create a new S3 bucket, or 'reuse' to re-use an existing bucket. The default is 'create'.
Default: create
Type: String
AllowedValues:
- create
- reuse
ConstraintDescription: "Must specify 'create' to create a new S3 bucket, or 'reuse' to re-use an existing bucket"
MyBucket:
Description: Please specify a name of the Amazon S3 bucket (irrespective if its new or existing). The name must be globally unique. Please see S3 Bucket naming rules https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucketnamingrules.html
Type: String
MinLength: 3
MaxLength: 63
ConstraintDescription: "Bucket names must be between 3 (min) and 63 (max) characters long, can consist only of lowercase letters, numbers, dots (.), and hyphens (-), must begin and end with a letter or number."
MyEmailAddress:
Description: SNS Subscription Email address to receive notifications from CloudWatch alarms of failed backups to S3. Must be a valid email address.
Type: String
MinLength: 5
AllowedPattern: '[^@]+@[^@]+\.[^@]+'
ConstraintDescription: "Must be a valid email address"
MetricName:
Description: The name of the CloudWatch metric that should be used. Can be any string. The default is "Backups".
Type: String
MinLength: 1
MaxLength: 255
Default: "Backups"
ConstraintDescription: "MetricName valid characters are A-Z, a-z, 0-9, “-” and “_” "
DimensionName:
Description: The name of the CloudWatch dimension that should be used for the above metric, based on what each S3 folder represents. E.g each folder can represent a System, Location or Customer, that you doing backups for. Can be any string. The default is "System".
Type: String
MinLength: 1
MaxLength: 255
Default: "System"
ConstraintDescription: "DimensionName valid characters are A-Z, a-z, 0-9, “-” and “_” "
BackupFrequencyPeriod:
Description: What schedule/frequency does your backups run, e.g. Hourly, daily, weekly? The value must be represented in seconds (minimum 60 seconds). The default is 86400 seconds = 1 day, i.e. you expect that files are backed up to S3 daily.
Type: Number
MinValue: 60
Default: 86400
ConstraintDescription: "BackupFrequencyPeriod must be in specified in seconds, with a minimum value of 60 (seconds)"
Conditions:
CreateS3Bucket: !Equals [!Ref CreateBucket, create]
ReuseS3Bucket: !Equals [!Ref CreateBucket, reuse]
Globals:
Function:
Timeout: 5
MemorySize: 128
Runtime: python3.12
CodeUri: monitor
Architectures:
- arm64 #Lambda functions will run on Arm-based AWS Graviton processors. The Powertools Lambda layer will also be Arm-based
Environment:
Variables:
POWERTOOLS_METRICS_NAMESPACE: BackupsMonitoring
#POWERTOOLS_SERVICE_NAME: S3Backups #TODO see if this fixes the CloudWatch Math Search, but misses Logging info
LOG_LEVEL: INFO
SNSTopic: !Ref EmailNotificationsSNSTopic
MetricName: !Ref MetricName
DimensionName: !Ref DimensionName
MyBucket: !Ref MyBucket
BackupFrequencyPeriod: !Ref BackupFrequencyPeriod
LoggingConfig:
LogGroup: !Sub /aws/lambda/${AWS::StackName}
LogFormat: JSON
Tags:
project: "monitor-backups-to-amazon-s3-using-amazon-cloudwatch"
Resources:
BackupBucket:
Type: AWS::S3::Bucket
Condition: CreateS3Bucket #only create this bucket if CreateBucket = create
DeletionPolicy: Retain
Properties:
BucketName: !Sub "${MyBucket}"
Tags:
- Key: project
Value: "monitor-backups-to-amazon-s3-using-amazon-cloudwatch"
CustomMetricsFunction:
Type: AWS::Serverless::Function
Properties:
Description: This Lambda function gets invoked each time a backup is copied to S3. It pushes a metric and dimension to CloudWatch for each backup
Handler: metrics.lambda_handler
Layers:
- !Sub arn:aws:lambda:${AWS::Region}:017000801446:layer:AWSLambdaPowertoolsPythonV2-Arm64:45 #The Powertools Lambda layer will also be Arm-based
Events:
BackupEvent:
Type: S3
Properties:
Bucket:
Ref: BackupBucket
Events: s3:ObjectCreated:*
AlarmsFunction:
Type: AWS::Serverless::Function
Properties:
Description: This Lambda function runs on a EventBridge schedule. It checks the metrics that exist (created by the CustomMetrics function), and creates CloudWatch Alarms for each metric.
Handler: alarms.lambda_handler
Layers:
- !Sub arn:aws:lambda:${AWS::Region}:017000801446:layer:AWSLambdaPowertoolsPythonV2-Arm64:45
Policies:
- Statement:
- Effect: "Allow"
Action:
- cloudwatch:PutMetricAlarm
- cloudwatch:ListMetrics
- cloudwatch:DescribeAlarmsForMetric
Resource:
- '*'
Events:
Schedule:
Type: ScheduleV2
Properties:
Description: run once a day, every day, starting based on when this schedule was created. So it will first run as soon as this stack is deployed, within a window of 15 minutes, and recur every day at that time.
ScheduleExpression: rate(24 hours)
ScheduleExpressionTimezone: GMT
FlexibleTimeWindow:
Mode: FLEXIBLE
MaximumWindowInMinutes: 15
HelperS3NotifcationFunction:
Type: AWS::Serverless::Function
Condition: ReuseS3Bucket #only create this function if CreateBucket = reuse
Properties:
Description: This Lambda function will create an Event Notification for the existing bucket to the CustomMetricsFunction Lambda
Handler: helper_s3notification.lambda_handler
Layers:
- !Sub arn:aws:lambda:${AWS::Region}:017000801446:layer:AWSLambdaPowertoolsPythonV2-Arm64:45
Environment:
Variables:
CustomMetricsFunctionArn: !GetAtt CustomMetricsFunction.Arn
Policies:
- Statement:
- Effect: "Allow"
Action:
- s3:PutBucketNotification
Resource:
- !Sub 'arn:aws:s3:::${MyBucket}'
CreateS3Notifcation:
Type: Custom::CreateS3NotifcationFunction #A CloudFormation custom resource, that runs on stack create, that runs the HelperS3NotifcationFunction
Condition: ReuseS3Bucket #only create this function if CreateBucket = reuse
Properties:
ServiceToken: !GetAtt HelperS3NotifcationFunction.Arn
EmailNotificationsSNSTopic:
Type: AWS::SNS::Topic
Properties:
Subscription:
- Endpoint: !Sub "${MyEmailAddress}"
Protocol: "email"
Tags:
- Key: project
Value: "monitor-backups-to-amazon-s3-using-amazon-cloudwatch"
CloudWatchDashboard:
Type: AWS::CloudWatch::Dashboard
Properties:
DashboardName: BackupsMonitoring
DashboardBody: !Sub "
{
\"periodOverride\": \"inherit\",
\"widgets\": [
{
\"type\":\"metric\",
\"x\":0,
\"y\":0,
\"width\":12,
\"height\":10,
\"properties\":{
\"metrics\":[
[ { \"expression\": \"FILL(SEARCH(' {BackupsMonitoring, ${DimensionName}} MetricName=\\\"${MetricName}\\\" ', 'Sum'), 0)\", \"id\": \"e1\" } ]
],
\"period\":${BackupFrequencyPeriod},
\"region\":\"${AWS::Region}\",
\"title\":\"Backups to S3\",
\"legend\": {
\"position\": \"right\"
}
}
}
]
}
"