Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: added tf infra for AWS ami account #6517

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions infra/aws/terraform/cncf-k8s-infra-aws-capa-ami/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# AWS Account for CAPA AMI Publication

This contains Terraform used to manage users & permissions for the **cncf-k8s-infra-aws-capa-ami** AWS account (`arn:aws:organizations::348685125169:account/o-kz4vlkihvy/819546954734`).

## Tool Requirements

* [Terraform](https://developer.hashicorp.com/terraform/downloads) v1.6.0 or greater
* AWS CLI

## Pre-reqs

This will need to be run by someone that is an admin in the account or by someone that can assume role to give admin in the account.

## Running

Set the AWS environment variables for the user that has access to the account.

Then run the following to disable the blocking of public AMIs:

```bash
hack/disable-block.sh
```

> NOTE: the script is used to disable the block as it doesn't naturally fit well into Terraform when running it across many regions.

Then do the usual terraform flow:

```bash
terraform init
terraform plan
terraform apply
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
#!/usr/bin/env bash

# Copyright 2024 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

## This script is used to disable the block on public access to AMIs

AMI_REGIONS="ap-south-1,eu-west-3,eu-west-2,eu-west-1,ap-northeast-2,ap-northeast-1,sa-east-1,ca-central-1,ap-southeast-1,ap-southeast-2,eu-central-1,us-east-1,us-east-2,us-west-1,us-west-2"

IFS=','
read -ra arr <<<"$AMI_REGIONS"

for val in "${arr[@]}"; do
aws ec2 disable-image-block-public-access --region "$val"
done
Comment on lines +24 to +26
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could do this with a HCL code ? cc @bryantbiggs

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_image_block_public_access

Don't have to be in this PR. Just want to be sure we can skip bash configuration

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm asking the other CAPA maintainer whether we need to continue publishing the AMIs to 15 regions. And whether we can just publish to 1 instead. If we change to just one then we can remove this completly.

But if there was a way that this could be done in HCL then that would give us options.

68 changes: 68 additions & 0 deletions infra/aws/terraform/cncf-k8s-infra-aws-capa-ami/iam.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
/*
Copyright 2024 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/


# NOTE: this is the standard permissions from the TF docs.
resource "aws_iam_policy" "imagebuilder" {
name = "capa-image-builder-policy"
path = "/"
description = "Recommended policy for image builder"

policy = jsonencode({
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:AttachVolume",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:CopyImage",
"ec2:CreateImage",
"ec2:CreateKeyPair",
"ec2:CreateSecurityGroup",
"ec2:CreateSnapshot",
"ec2:CreateTags",
"ec2:CreateVolume",
"ec2:DeleteKeyPair",
"ec2:DeleteSecurityGroup",
"ec2:DeleteSnapshot",
"ec2:DeleteVolume",
"ec2:DeregisterImage",
"ec2:DescribeImageAttribute",
"ec2:DescribeImages",
"ec2:DescribeInstances",
"ec2:DescribeInstanceStatus",
"ec2:DescribeRegions",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSnapshots",
"ec2:DescribeSubnets",
"ec2:DescribeTags",
"ec2:DescribeVolumes",
"ec2:DetachVolume",
"ec2:GetPasswordData",
"ec2:ModifyImageAttribute",
"ec2:ModifyInstanceAttribute",
"ec2:ModifySnapshotAttribute",
"ec2:RegisterImage",
"ec2:RunInstances",
"ec2:StopInstances",
"ec2:TerminateInstances"
],
"Resource": "*"
}
]
})
}
37 changes: 37 additions & 0 deletions infra/aws/terraform/cncf-k8s-infra-aws-capa-ami/oidc.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
/*
Copyright 2024 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

module "iam_github_oidc_provider" {
source = "terraform-aws-modules/iam/aws//modules/iam-github-oidc-provider"

client_id_list = [
"sts.amazonaws.com",
]

tags = var.tags
}

module "iam_github_oidc_role" {
source = "terraform-aws-modules/iam/aws//modules/iam-github-oidc-role"

name = "gh-image-builder"
subjects = ["kubernetes-sigs/cluster-api-provider-aws:*"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a particular reason why we are using github actions instead of prow? We can configure prow jobs to assume AWS roles very easily

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we started this work in was advised to use TF. Do we use IRSA or pod identity with any Prow jobs at present?

Copy link
Contributor Author

@richardcase richardcase Sep 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having a quick look i only see assume roles in a few jobs and these are "trusted" jobs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do, with kops and capz.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent, i will hunt them out. Thanks @upodroid

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@upodroid - any chance you could point me at one of the jobs? I'm not really seeing how this is done from Prow.

I can see this trusted job: https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes/sig-k8s-infra/trusted/sig-k8s-infra-registry.yaml#L32

policies = {
ImageBuilder = aws_iam_policy.imagebuilder.arn
}

tags = var.tags
}
44 changes: 44 additions & 0 deletions infra/aws/terraform/cncf-k8s-infra-aws-capa-ami/providers.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
/*
Copyright 2024 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

terraform {
backend "s3" {
bucket = "cncf-k8s-infra-aws-capa-ami-tfstate"
Copy link
Member

@ameukam ameukam Sep 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In which account this bucket is created ? I assume we want to manually create this ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know to be honest. Which account are the TF buckets usually created in?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We usually keep the remote backend in the same AWS account. There is a pending effort to centralize the remote backends in a dedicated account. I'll manually create this bucket for this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thank you @ameukam

key = "terraform.tfstate"
region = "us-east-2"
}

required_version = "~> 1.8.0"

required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.66"
}
}
}

provider "aws" {
region = var.region

assume_role {
role_arn = "arn:aws:iam::348685125169:role/OrganizationAccountAccessRole"
}

default_tags {
tags = var.tags
}
}
31 changes: 31 additions & 0 deletions infra/aws/terraform/cncf-k8s-infra-aws-capa-ami/variables.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
/*
Copyright 2024 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

variable "tags" {
type = map(string)
default = {
"managed-by" = "Terraform",
"group" = "sig-cluster-lifecycle",
"subproject" = "cluster-api-provider-aws"
"githubRepo" = "git.k8s.io/k8s.io"
}
}

variable "region" {
type = string
description = "AWS region for region specific resources"
default = "us-east-2"
}