Multi-tenancy considerations

[pdf]

Hello!

• Vote on this issue by adding a 👍 reaction
• If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details

The current CRDs do not provide a method of exposing Pulumi programs/stacks to unprivileged users/processes.

An example scenario of where this is problematic:

We wish to be able to allow unprivileged users to create/destroy S3 buckets with a templated name, and obtain bucket-specific credentials to use within their deployments.

The IAM privileges required by the operator for these operations are quite significant, but given the design of the Stack/Program CRDs there is no way to limit the operations that a user might perform if we allow them to write these resources directly.

Proposed solution

This is not a formal proposal, but may serve as a starting point for discussion.

One possible path to providing the capability to expose specific stacks/programs would be to introduce a CRD-generating meta resource definition, consuming that CRD that might look something like this:

---
apiVersion: pulumi.com/v1
kind: StackResource
metadata:
  name: s3bucket
spec:
  group: my.group.org
  names:
    kind: s3bucket
  versions:
    - name: v1
      config:
        region:
          type: String
      template:
        metadata:
          name: ${claim.metadata.namespace}-${claim.metadata.name}
          namespace: operator-namespace
          labels: ${claim.metadata.labels}
        spec:
          stack: <YOUR ORG>/s3bucket/${claim.metadata.namespace}-${claim.metadata.name}
          programRef:
            name: bucketoprivs
          destroyOnFinalize: true
          config:
            aws:region: ${claim.spec.config.region}
            claim:name: ${claim.metadata.name}
            other:key: static-value

This cribs some properties from regular CRD spec, with the idea that upon creating such a CR the operator would generate a CRD based on the provided properties. The template field of the version would be used by the operator to create a Stack CR in the operator NS whenever a CR for the generated CRD is created (and bound to the lifecycle of the user CR), which should allow the existing operator logic to continue operating on stacks without modification. A user would consume the generated CRD like so:

---
apiVersion: my.group.org/v1
kind: s3bucket
metadata:
  name: stackname
spec:
  config:
    region: us-east-1

By generating this intermediate CRD we can allow users to consume Stacks/Programs without exposing the full power/privileges of the operator, and control access using standard k8s RBAC.

I'm hand-waving away a lot of details here, but can try to allocate some more design time if this is an approach that seems sensible.

[cleverguy25]

Added to epic #586

[EronWright]

Good news everyone, we just release a preview of Pulumi Kubernetes Operator v2. This new release has a whole-new architecture that provides much better isolation between stacks. Regarding how to support low-privilege users, the key is that each stack is associated with a Kubernetes service account. If your program creates Kubernetes resources, that service account is the identity for those operations, and you control the role binding(s). The operator's own privileges aren't inherited by the stack as was the case in v1.

Please read the announcement blog post for more information:
https://www.pulumi.com/blog/pulumi-kubernetes-operator-2-0/

Would love to hear your feedback! Feel free to engage with us on the #kubernetes channel of the Pulumi Slack workspace.
cc @pdf @DominikSchorn

[pdf]

@EronWright the structure in v2 looks like a good first step as it can significantly narrow the operating scope of credentials, but I don't believe this solves the exemplar problem statement from the original issue here:

We wish to be able to allow unprivileged users to create/destroy S3 buckets with a templated name, and obtain bucket-specific credentials to use within their deployments.

The IAM privileges required by the operator for these operations are quite significant, but given the design of the Stack/Program CRDs there is no way to limit the operations that a user might perform if we allow them to write these resources directly.

Unless I'm missing something, the problem still exists with the operator v2 pattern since we must still allow users to create Stack resources, and we must give those users access to a ServiceAccount that has sufficient permissions to perform all of the above actions.

The issue is that IAM permissions may not be granular enough to allow restricting operations to effect only specific entities, and since we must allow users to create Stacks, they're able to execute arbitrary code using the granted privileges.

The reason I suggested the abstraction in the OP is that it would allow consumers of a Stack resource to deploy instances of the stack without being allowed to define new Stack resources themselves, and without having direct access to the privileges that allow the stack to operate.

---

PS - I know you suggested reaching out on Slack, but I'm UTC+11 so async is likely easier to manage