Official "build environment" image for building new Pulumi providers

[christophermaier]

Hello!

• Vote on this issue by adding a 👍 reaction
• If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details

I'm currently in the process of converting a handful of Terraform providers to Pulumi providers, using the pulumi-tf-provider-boilerplate as a starting point.

In order to avoid having to install Go, NodeJS, .NET, and Python runtimes in my CI agents¹, I'm running build steps in a container that has them all. Naturally, I started with the official pulumi/pulumi image since it has all that already. However, this image does not have everything needed for building providers and SDKs, as currently implemented in the pulumi-tf-provider-boilerplate repository. In particular, it is missing pulumictl, golangci-lint, and goreleaser. Additionally, the pulumi/pulumi image is clearly intended for running pulumi (its ENTRYPOINT is pulumi), and thus requires additional modifications in order to be used as a build container in CI environments (or on developer workstations, though that is not my personal concern in this issue at this time).

For my own use, I have created the following Dockerfile:

# syntax=docker/dockerfile:1.3-labs

ARG PULUMI_VERSION

# This gets us the Pulumi CLI, as well as current language runtimes
# for Go, Node/TypeScript, Python, and .NET.
FROM pulumi/pulumi:${PULUMI_VERSION} as base

ARG PULUMICTL_VERSION
ARG GOLANGCI_LINT_VERSION
ARG GORELEASER_VERSION

RUN <<EOF
curl \
    --proto "=https" \
    --tlsv1.2 \
    --location \
    --fail \
    --verbose \
    --output "pulumictl.tar.gz" \
    "https://github.com/pulumi/pulumictl/releases/download/v${PULUMICTL_VERSION}/pulumictl-v${PULUMICTL_VERSION}-linux-amd64.tar.gz"
mkdir pulumictl_extraction
tar --extract --gunzip --verbose --directory pulumictl_extraction --file pulumictl.tar.gz
mv pulumictl_extraction/pulumictl /usr/local/bin/pulumictl
chmod a+x /usr/local/bin/pulumictl
rm -Rf pulumictl_extraction
rm pulumictl.tar.gz

# Install golangci-lint
curl --proto "=https" \
    --tlsv1.2 \
    --silent \
    --show-error \
    --fail \
    --location \
    https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh \
| sh -s -- -b $(go env GOPATH)/bin v${GOLANGCI_LINT_VERSION}

# Install goreleaser
go install github.com/goreleaser/goreleaser@v${GORELEASER_VERSION}

EOF

# The default entrypoint of our base image is `pulumi`; we don't
# want that.
ENTRYPOINT []
CMD ["bash"]

It would be useful to have an official image along these lines, however, as I feel this would make it easier for people to contribute new Pulumi providers, and would not require everyone to "reinvent the wheel". This is particularly true for developers that may not have familiarity with some of the language ecosystems that Pulumi works in (at my company, we work with Python and NodeJS, but not Go or .NET, for instance). Such an image need not be the existing pulumi/pulumi image (these seem like related, but distinct usecases, after all), or even based on it; I simply took this approach for expediency.

Thanks! 🙇

Footnotes

1. While I am aware of the Github Action workflows that Pulumi has set up for these kind of providers, which obviate the need for build containers like this, not everyone uses (or wants to use) Github Actions, so a more platform-agnostic solution would help with broader adoption. We use Buildkite for CI, for what it's worth, though other CI platforms would have similar needs. ↩

[mikhailshilkov]

cc @stack72

[RobbieMcKinstry]

Hi @christophermaier . This request sounds like something we'd be able to accept in one form or another.

1. I think we could probably install golangci-lint, pulumictl, and goreleaser in the Kitchen-Sink binary and call it a day. That doesn't entirely solve your initial use case, however, since you'd like to modify the entrypoint too so your CI system can more easily build providers.

2. Another solution would be to offer a "tf-provider" Docker base image, which is essentially what you've provided above. I can see how this would be useful. Would you be willing to submit a PR?

I'm moving this issue to the Docker Container repo.

[christophermaier]

@RobbieMcKinstry Sorry for the delay; yes, I'd be happy to submit a PR with this. Looking at the existing Dockerfiles here, it looks like the most direct path forward would be to add an additional build target to the all-in-one pulumi Dockerfile, since that's essentially what my image is already. If that sounds OK with you, I can set it up.

Thanks!

[RobbieMcKinstry]

I think that works for us! Please be aware of this open issue that will result in CI build failures.

Thanks very much for offering your assistance! :)