Skip to content

Commit

Permalink
Grant S3 access to notebook & dask pods
Browse files Browse the repository at this point in the history
eksctl
[supports](https://eksctl.io/usage/iamserviceaccounts/#usage-with-config-files)
creating kubernetes service acocunts bound with
[IRSA](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html).
We create one with S3 access, and bind it to our notebook and dask
pods. This should give them full s3 access.

Remove separate eksctl cluster jsonnet object, since it was
not doing anything useful.

Stolen from 2i2c-org#436

Fixes 2i2c-org#492
  • Loading branch information
yuvipanda committed Aug 19, 2021
1 parent 35a830b commit d8e3d73
Show file tree
Hide file tree
Showing 3 changed files with 31 additions and 22 deletions.
8 changes: 8 additions & 0 deletions config/hubs/carbonplan.cluster.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ hubs:
name: Carbon Plan
url: https://carbonplan.org
singleuser:
serviceAccountName: cloud-user-sa
initContainers:
# Need to explicitly fix ownership here, since EFS doesn't do anonuid
- name: volume-mount-ownership-fix
Expand Down Expand Up @@ -168,6 +169,13 @@ hubs:
admin_users: *users

dask-gateway:
backend:
scheduler:
extraPodConfig:
serviceAccountName: cloud-user-sa
worker:
extraPodConfig:
serviceAccountName: cloud-user-sa
traefik:
resources:
requests:
Expand Down
26 changes: 23 additions & 3 deletions eksctl/carbonplan.jsonnet
Original file line number Diff line number Diff line change
@@ -1,12 +1,16 @@
// Exports an eksctl config file for carbonplan cluster
local cluster = import "./libsonnet/cluster.jsonnet";
local ng = import "./libsonnet/nodegroup.jsonnet";

// place all cluster nodes here
local clusterRegion = "us-west-2";
local masterAzs = ["us-west-2a", "us-west-2b", "us-west-2c"];
local nodeAz = "us-west-2a";

// List of namespaces where we have hubs deployed
// Each will get a ServiceAccount that will get credentials to talk
// to AWS services, via https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html
local namespaces = ['staging', 'prod'];

// Node definitions for notebook nodes. Config here is merged
// with our notebook node definition.
// A `node.kubernetes.io/instance-type label is added, so pods
Expand All @@ -33,12 +37,28 @@ local daskNodes = [
{ instancesDistribution+: { instanceTypes: ["r5.8xlarge"] }},
];

cluster {
{
apiVersion: 'eksctl.io/v1alpha5',
kind: 'ClusterConfig',
metadata+: {
name: "carbonplanhub",
region: clusterRegion
region: clusterRegion,
version: '1.19'
},
availabilityZones: masterAzs,
iam: {
withOIDC: true,

serviceAccounts: [{
metadata: {
name: "cloud-user-sa",
namespace: namespace
},
attachPolicyARNs:[
"arn:aws:iam::aws:policy/AmazonS3FullAccess"
],
} for namespace in namespaces],
},
nodeGroups: [
ng {
name: 'core-a',
Expand Down
19 changes: 0 additions & 19 deletions eksctl/libsonnet/cluster.jsonnet

This file was deleted.

0 comments on commit d8e3d73

Please sign in to comment.