Add basic OpenShift support and setup [Part 1] #1955

tnozicka · 2024-06-05T17:59:09Z

Description of your changes:
This PR brings the initial support for running on OpenShift.

It also fixes the operator when run on platforms where OwnerReferencesPermissionEnforcement is enabled.

It also moves all persistent storage to /var/lib which is available on all OSes, including atomic ones / CoreOS.

Note: This is part 1 that wires everything together and fixes most of the issues. Some tests that fail only on OpenShift still fail and need individual followups and we also needs to wire in proper cpu limits that flake a lot of tests waiting for certs to be generated.

Which issue is resolved by this Pull Request:
Resolves #713 #1935

scylla-operator-bot · 2024-06-05T17:59:12Z

@tnozicka: GitHub didn't allow me to request PR reviews from the following users: tnozicka.

Note that only scylladb members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

Description of your changes:
TODO

Which issue is resolved by this Pull Request:
Resolves #713 #1935

/cc

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

scylla-operator-bot · 2024-06-05T17:59:21Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tnozicka

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [tnozicka]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

hack/ci-deploy.sh

further

tnozicka · 2024-12-27T15:22:56Z

/test e2e-openshift-aws-parallel
/test e2e-openshift-aws-serial
(out of interest, but expected to fail on some)

hack/.ci/run-e2e-openshift-aws.sh

rzetelskik · 2024-12-27T15:42:08Z

hack/.ci/run-e2e-openshift-aws.sh

+export SCYLLA_OPERATOR_FEATURE_GATES
+
+for i in "${!KUBECONFIGS[@]}"; do
+  KUBECONFIG="${KUBECONFIGS[$i]}" DEPLOY_DIR="${ARTIFACTS}/deploy/${i}" timeout --foreground -v 10m "${parent_dir}/../ci-deploy.sh" "${SO_IMAGE}" &


we should either add a dedicated script running ci-deploy-release.sh and set it up in CI or at the very least have a tracking issue

for openshift? the manifests are shared

Is the split for GKE only coming from the fact ci-deploy.sh was there before the periodics or is there a different reason that's not there for openshift? In other words why would GKE periodics run a different script than the OpenShift ones (GKE periodics run ci-deploy-release.sh through run-e2e-gke-release.sh, while this runs ci-deploy.sh for both presubmits and periodics).

ci-deploy-release.sh uses the manifests from a real commit (which is not there for PRs because it's a local merge, so they use ci-deploy.sh, by design.

run-e2e-openshift-aws.sh is the entrypoint for presubmits (actually wired for periodics as well at this point)
run-e2e-openshift-aws-release.sh needs to be added later on and wired for postsubmits and periodics

helm/scylla-operator/templates/scylladbmonitoring_grafana_clusterrole_def_openshift.yaml

helm/scylla-operator/templates/scylladbmonitoring_prometheus_clusterrole_def_openshift.yaml

test/e2e/set/nodeconfig/nodeconfig_disksetup.go

rzetelskik · 2024-12-27T16:02:02Z

Also please make the PR title a bit more descriptive

tnozicka · 2024-12-27T19:05:27Z

/test e2e-openshift-aws-serial
(cluster provisioning failed)

tnozicka · 2024-12-27T21:12:31Z

/retest
e2e-gke-release-script-latest should be unborked now

tnozicka · 2024-12-27T21:13:50Z

/test e2e-openshift-aws-parallel
/test e2e-openshift-aws-serial
(for reference - some of the failures are races / easy wins for the next PR)

scylla-operator-bot · 2024-12-27T21:29:28Z

@tnozicka: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-openshift-aws-serial	`f0db302`	link	false	`/test e2e-openshift-aws-serial`
ci/prow/e2e-openshift-aws-parallel	`f0db302`	link	false	`/test e2e-openshift-aws-parallel`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

tnozicka · 2024-12-27T21:49:54Z

ci/prow/e2e-gke-release-script-latest

this has to fail because it deploys local-csi-driver from a released image and we have changed the mount paths

https://prow.scylla-operator.scylladb.com/view/gs/scylla-operator-prow/pr-logs/pull/scylladb_scylla-operator/1955/pull-scylla-operator-master-e2e-gke-release-script-latest/1872752453217161216#1:test-build-log.txt%3A287

note: 'MountVolume.SetUp failed for volume "volumes-dir" : hostPath type check failed:
  /mnt/persistent-volumes is not a directory'

https://gcsweb.scylla-operator.scylladb.com/gcs/scylla-operator-prow/pr-logs/pull/scylladb_scylla-operator/1955/pull-scylla-operator-master-e2e-gke-release-script-latest/1872752453217161216/artifacts/must-gather/0/namespaces/local-csi-driver/events.events.k8s.io/local-csi-driver-jtsx9.1815258d6c62fdda.yaml

/override ci/prow/e2e-gke-release-script-latest

scylla-operator-bot · 2024-12-27T21:49:57Z

@tnozicka: Overrode contexts on behalf of tnozicka: ci/prow/e2e-gke-release-script-latest

In response to this:

ci/prow/e2e-gke-release-script-latest

this has to fail because it deploys local-csi-driver from a released image and we have changed the mount paths

https://prow.scylla-operator.scylladb.com/view/gs/scylla-operator-prow/pr-logs/pull/scylladb_scylla-operator/1955/pull-scylla-operator-master-e2e-gke-release-script-latest/1872752453217161216#1:test-build-log.txt%3A287
note: 'MountVolume.SetUp failed for volume "volumes-dir" : hostPath type check failed:
 /mnt/persistent-volumes is not a directory'
https://gcsweb.scylla-operator.scylladb.com/gcs/scylla-operator-prow/pr-logs/pull/scylladb_scylla-operator/1955/pull-scylla-operator-master-e2e-gke-release-script-latest/1872752453217161216/artifacts/must-gather/0/namespaces/local-csi-driver/events.events.k8s.io/local-csi-driver-jtsx9.1815258d6c62fdda.yaml

/override ci/prow/e2e-gke-release-script-latest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

rzetelskik · 2024-12-28T00:20:56Z

(one outstanding comment)

tnozicka added kind/feature Categorizes issue or PR as related to a new feature. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. labels Jun 5, 2024

scylla-operator-bot bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 5, 2024

scylla-operator-bot bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 5, 2024

tnozicka force-pushed the fix-openshift branch 2 times, most recently from 0afcbff to 3b42f12 Compare June 14, 2024 10:09

tnozicka marked this pull request as draft June 14, 2024 11:14

tnozicka force-pushed the fix-openshift branch from 3b42f12 to 5b15d2b Compare June 24, 2024 13:13

scylla-operator-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 24, 2024

tnozicka force-pushed the fix-openshift branch 3 times, most recently from a9522f7 to 3fd195c Compare June 28, 2024 12:45

scylla-operator-bot bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 9, 2024

tnozicka force-pushed the fix-openshift branch from 3fd195c to d0ccf55 Compare July 19, 2024 07:48

scylla-operator-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jul 19, 2024

tnozicka force-pushed the fix-openshift branch 3 times, most recently from 75a8f53 to 8513810 Compare July 26, 2024 14:57

tnozicka marked this pull request as ready for review August 7, 2024 11:43

tnozicka force-pushed the fix-openshift branch 2 times, most recently from 0a08384 to 1475312 Compare August 9, 2024 09:09

rzetelskik reviewed Aug 9, 2024

View reviewed changes

hack/ci-deploy.sh Outdated Show resolved Hide resolved

scylladb deleted a comment from scylla-operator-bot bot Aug 9, 2024

tnozicka force-pushed the fix-openshift branch 4 times, most recently from 1dccbce to 7a2ea0e Compare December 27, 2024 13:54

tnozicka added 3 commits December 27, 2024 15:55

Fix scyllaoperatorconfig error handling

f52417a

Use e2e storageClass with monitoring fixtures

8350450

Grant e2e sa ScyllaCluster member permissions so it can grant them

ed87226

further

tnozicka force-pushed the fix-openshift branch 2 times, most recently from 5800971 to 169c28a Compare December 27, 2024 15:12

tnozicka changed the title ~~[WIP] Fix OpenShift [Part 1]~~ Fix OpenShift [Part 1] Dec 27, 2024

scylla-operator-bot bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 27, 2024

tnozicka force-pushed the fix-openshift branch from 169c28a to 35140a2 Compare December 27, 2024 15:22

rzetelskik reviewed Dec 27, 2024

View reviewed changes

tnozicka changed the title ~~Fix OpenShift [Part 1]~~ Add basic OpenShift support [Part 1] Dec 27, 2024

tnozicka added 2 commits December 27, 2024 19:38

Add basic OpenShift support, permissions and path fixes

9d36ac3

Update generated

f0db302

tnozicka force-pushed the fix-openshift branch from 35140a2 to f0db302 Compare December 27, 2024 18:38

tnozicka changed the title ~~Add basic OpenShift support [Part 1]~~ Add basic OpenShift support and setup[Part 1] Dec 27, 2024

tnozicka changed the title ~~Add basic OpenShift support and setup[Part 1]~~ Add basic OpenShift support and setup [Part 1] Dec 27, 2024

tnozicka mentioned this pull request Dec 27, 2024

Add an e2e test for retryable NodeConfig mount units #2253

Open

tnozicka requested a review from rzetelskik December 27, 2024 21:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add basic OpenShift support and setup [Part 1] #1955

Add basic OpenShift support and setup [Part 1] #1955

tnozicka commented Jun 5, 2024 •

edited

Loading

scylla-operator-bot bot commented Jun 5, 2024

scylla-operator-bot bot commented Jun 5, 2024

tnozicka commented Dec 27, 2024

rzetelskik Dec 27, 2024

tnozicka Dec 27, 2024

rzetelskik Dec 27, 2024 •

edited

Loading

tnozicka Dec 28, 2024 •

edited

Loading

rzetelskik commented Dec 27, 2024

tnozicka commented Dec 27, 2024

tnozicka commented Dec 27, 2024

tnozicka commented Dec 27, 2024 •

edited

Loading

scylla-operator-bot bot commented Dec 27, 2024 •

edited

Loading

tnozicka commented Dec 27, 2024

scylla-operator-bot bot commented Dec 27, 2024

rzetelskik commented Dec 28, 2024

Add basic OpenShift support and setup [Part 1] #1955

Are you sure you want to change the base?

Add basic OpenShift support and setup [Part 1] #1955

Conversation

tnozicka commented Jun 5, 2024 • edited Loading

scylla-operator-bot bot commented Jun 5, 2024

scylla-operator-bot bot commented Jun 5, 2024

tnozicka commented Dec 27, 2024

rzetelskik Dec 27, 2024

Choose a reason for hiding this comment

tnozicka Dec 27, 2024

Choose a reason for hiding this comment

rzetelskik Dec 27, 2024 • edited Loading

Choose a reason for hiding this comment

tnozicka Dec 28, 2024 • edited Loading

Choose a reason for hiding this comment

rzetelskik commented Dec 27, 2024

tnozicka commented Dec 27, 2024

tnozicka commented Dec 27, 2024

tnozicka commented Dec 27, 2024 • edited Loading

scylla-operator-bot bot commented Dec 27, 2024 • edited Loading

tnozicka commented Dec 27, 2024

scylla-operator-bot bot commented Dec 27, 2024

rzetelskik commented Dec 28, 2024

tnozicka commented Jun 5, 2024 •

edited

Loading

rzetelskik Dec 27, 2024 •

edited

Loading

tnozicka Dec 28, 2024 •

edited

Loading

tnozicka commented Dec 27, 2024 •

edited

Loading

scylla-operator-bot bot commented Dec 27, 2024 •

edited

Loading