Can't reliably deploy in 2024

[dantecl]

Hey folks,

I can't deploy this project using tag v2.6.0 because it has a python37 constraint, appengine barks at that. Deploying from master using python311 goes much further but has completely undocumented requirements like the terraform/k8s code that doesn't work out of the box. Do you guys have some more recent documentation to add to the project that we non-googlers could deploy to GCP?

[vitorguidi]

@dantecl can you drill down on what exactly does not work out of the box, as far as K8S and Terraform go?

[dantecl]

When I generate the config, none of the terraform code gets copied over. If I copy it manually, I have to fill out the variables file, and then the deploy process does not actually create any of the resources. I've resorted to deploying it with --target appengine to bypass the terraform stage, and most of the cronjobs in appengine fail with 404 to the cron-service service.

[vitorguidi]

Re crons, they are supposed to run on Kubernetes, so it is not expected that appengine would succeed.

As far as the overall deployment goes, we have a pending project that will replace butler.py for terraform while bootstrapping the infrastructure, so I suggest you follow #3788. Since the deployment strategy will change, docs will be out once this lands.

[dantecl]

Also, on the terraform deployment, with -target=module.clusterfuzz it does not create anything, and if I remove it, it does "Plan: 7 to add, 0 to change, 0 to destroy" with it trying to clobber my existing VPC, subnet and NAT gateway. I'll follow #3788 for updates, do you have any idea on timeframes?

[github-actions]

This issue has not had any activity for 60 days and will be automatically closed in two weeks

[varseand]

I second @dantecl request. I've moved further in deployment by crafting my own config based on infra/k8s and infra/terraform but there're more issues coming:

• some indexes are not added to src/appengine/index.yaml (e.g. WindowRateLimitTask)
• some services are not activated at the time of create_config (e.g. secrets manager)
• secrets (gcs-signer-key) are not added at create_config stage

Got stuck on the last problem as I'm not sure what the key should be.

@vitorguidi @jonathanmetzman are there any timelines on switching to new deployment scripts? Anything we could help with to let the project become deployable again for new setups?

[vitorguidi]

Hey there.

I am currently working in bootstrapping a development environment for our own use. This will probably help out with these deployment pains.

As far as helping us out, please document all the problems you are facing in this issue. If you end up solving stuff on your own, please let us know how you did it.

Re timelines, it is hard to estimate a completion date because this is a section of the system I am unfamiliar with, but there is active effort in this problem right now.

[vitorguidi]

Work on create config will be here > #4724

@varseand

[vitorguidi]

I second @dantecl request. I've moved further in deployment by crafting my own config based on infra/k8s and infra/terraform but there're more issues coming:

• some indexes are not added to src/appengine/index.yaml (e.g. WindowRateLimitTask)
• some services are not activated at the time of create_config (e.g. secrets manager)
• secrets (gcs-signer-key) are not added at create_config stage

Got stuck on the last problem as I'm not sure what the key should be.

@vitorguidi @jonathanmetzman are there any timelines on switching to new deployment scripts? Anything we could help with to let the project become deployable again for new setups?

Re the secret, it is supposed to contain the json service account token from a service account with GCS permissions, so that the presign can happen during preprocess stage of tasks.

Re the index, the only one that I saw fail so far is the window rate limit task you mentioned. I added it in the PR I mentioned and the trouble went away.

This feedback on what goes wrong when you try to deploy is important to us, please send more our way as you go =)

[github-actions]

This issue has not had any activity for 60 days and will be automatically closed in two weeks

[vitorguidi]

I managed to bootstrap ClusterFuzz with the changes in this PR > #4793

The desirable state is still to have a Terraform module, but this will only be done later this year. Meanwhile, this will achieve the same goal. @varseand all the concerns you raised in that previous comment are addressed

[bkosciarz]

@vitorguidi Thanks for getting this in. Does it make sense to tag a new release of Clusterfuzz now? #4709

[vitorguidi]

@vitorguidi Thanks for getting this in. Does it make sense to tag a new release of Clusterfuzz now? #4709

We haven't done releases in this way for a while, so the policy ATM is to deploy the state of master. @jonathanmetzman has more context on this, and if we will do releases again the near future