Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

controllers: fix issues found in recent testing #82

Merged
merged 1 commit into from
Aug 2, 2024

Conversation

leelavg
Copy link
Contributor

@leelavg leelavg commented Aug 2, 2024

  1. While setting SELinuxMount on CSIDriver we need to check for existence of value set in Nodeplugin before using it or else CSIDriver will reconile infinitely due to a spurious diff
(dlv) p desiredCsiDriver.Spec
k8s.io/api/storage/v1.CSIDriverSpec {
        AttachRequired: *true,
        PodInfoOnMount: *false,
        VolumeLifecycleModes: []k8s.io/api/storage/v1.VolumeLifecycleMode len: 1, cap: 1, [
                "Persistent",
        ],
        StorageCapacity: *false,
        FSGroupPolicy: *"File",
        TokenRequests: []k8s.io/api/storage/v1.TokenRequest len: 0, cap: 0, nil,
        RequiresRepublish: *false,
        SELinuxMount: *bool nil,} <=====>
(dlv) p existingCsiDriver.Spec
k8s.io/api/storage/v1.CSIDriverSpec {
        AttachRequired: *true,
        PodInfoOnMount: *false,
        VolumeLifecycleModes: []k8s.io/api/storage/v1.VolumeLifecycleMode len: 1, cap: 1, [
                "Persistent",
        ],
        StorageCapacity: *false,
        FSGroupPolicy: *"File",
        TokenRequests: []k8s.io/api/storage/v1.TokenRequest len: 0, cap: 0, nil,
        RequiresRepublish: *false,
        SELinuxMount: *false,} <=====>
  1. We need to mount csi configmap in Nodeplugin as well
  2. Topology feature gate should be false by default
  3. Fix typo in CSI plugin roles

@leelavg
Copy link
Contributor Author

leelavg commented Aug 2, 2024

Below is the testing result when I rebased this PR on #48. I raised #79, #80, #81 to further refine the operations which I couldn't capture in this PR.

PVC provisioning, snapshot, restore, clone.

> ./workload.py
2024-08-02 11:46:30,847 INFO     StatefulSet/ubi9-fs is created
2024-08-02 11:46:30,847 INFO     Waiting for StatefulSet/ubi9-fs to become ready
2024-08-02 12:13:13,887 INFO     StatefulSet/ubi9-fs-clone is created
2024-08-02 12:13:13,887 INFO     Waiting for StatefulSet/ubi9-fs-clone to become ready
2024-08-02 12:13:26,965 INFO     VolumeSnapshot/cephfs-snap is created
2024-08-02 12:13:27,425 INFO     StatefulSet/ubi9-fs-restore is created
2024-08-02 12:13:27,425 INFO     Waiting for StatefulSet/ubi9-fs-restore to become ready
2024-08-02 12:14:00,542 INFO     StatefulSet/ubi9-bk is created
2024-08-02 12:14:00,542 INFO     Waiting for StatefulSet/ubi9-bk to become ready
2024-08-02 12:14:16,181 INFO     StatefulSet/ubi9-bk-clone is created
2024-08-02 12:14:16,181 INFO     Waiting for StatefulSet/ubi9-bk-clone to become ready
2024-08-02 12:14:29,274 INFO     VolumeSnapshot/rbd-snap is created
2024-08-02 12:14:29,753 INFO     StatefulSet/ubi9-bk-restore is created
2024-08-02 12:14:29,754 INFO     Waiting for StatefulSet/ubi9-bk-restore to become ready
2024-08-02 12:14:48,872 INFO     Writing logs from workload pods to ./workload.log

> bat workload.log
───────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: workload.log
───────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ 2024-07-15T14:10:46.217368 ubi9-fs-0 ['Old:', 'New:', 'b80dc4b5ed5597f633f0bdc40ad1130d  -']
   2   │ 2024-07-15T14:10:46.740037 ubi9-fs-clone-0 ['Old:', 'b80dc4b5ed5597f633f0bdc40ad1130d  -', 'New:', 'bd020f952eef223fb8b32978f27bebec  -']
   3   │ 2024-07-15T14:10:47.266860 ubi9-fs-restore-0 ['Old:', 'bd020f952eef223fb8b32978f27bebec  -', 'New:', '798bda8433047e84f736cc80616696b3  -']
   4   │ 2024-07-15T14:10:47.788266 ubi9-bk-0 ['Old:', 'f1c9645dbc14efddc7d8a322685f26eb  -', 'New:', 'b3b423d3677b8aa83666dbd1f07b87bb  -']
   5   │ 2024-07-15T14:10:48.308433 ubi9-bk-clone-0 ['Old:', 'b3b423d3677b8aa83666dbd1f07b87bb  -', 'New:', '5795ac08eba535ce71f6e0dd46e13760  -']
   6   │ 2024-07-15T14:10:48.829621 ubi9-bk-restore-0 ['Old:', '5795ac08eba535ce71f6e0dd46e13760  -', 'New:', '3dba6b88d2ce44669746caef758bfa2d  -']
   7   │ 2024-08-02T12:14:49.729588 ubi9-fs-0 ['Old:', 'New:', 'e15958a59c73abdfd435fd9f197570ba  -']
   8   │ 2024-08-02T12:14:50.238487 ubi9-fs-clone-0 ['Old:', 'e15958a59c73abdfd435fd9f197570ba  -', 'New:', '438a066e650c73a76faf27501cf31a09  -']
   9   │ 2024-08-02T12:14:50.747487 ubi9-fs-restore-0 ['Old:', '438a066e650c73a76faf27501cf31a09  -', 'New:', '321b2e0b705827b658fdaca8b6f81965  -']
  10   │ 2024-08-02T12:14:51.256788 ubi9-bk-0 ['Old:', 'f1c9645dbc14efddc7d8a322685f26eb  -', 'New:', 'dc265a771eda5ddec2e8955029ebad63  -']
  11   │ 2024-08-02T12:14:51.760383 ubi9-bk-clone-0 ['Old:', 'dc265a771eda5ddec2e8955029ebad63  -', 'New:', 'f9dfa12b74bdc6bd166f087d517cc3f3  -']
  12   │ 2024-08-02T12:14:52.265203 ubi9-bk-restore-0 ['Old:', 'f9dfa12b74bdc6bd166f087d517cc3f3  -', 'New:', '39c0cf06ee096871368cc66ed6c8a433  -']
───────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

config/rbac/csi_cephfs_ctrlplugin_role.yaml Outdated Show resolved Hide resolved
config/rbac/csi_rbd_ctrlplugin_role.yaml Outdated Show resolved Hide resolved
config/rbac/csi_rbd_nodeplugin_role.yaml Outdated Show resolved Hide resolved
internal/controller/driver_controller.go Outdated Show resolved Hide resolved
1. While setting SELinuxMount on CSIDriver we need to check for existence
of value set in Nodeplugin before using it or else CSIDriver will
reconile infinitely due to a spurious diff
```
(dlv) p desiredCsiDriver.Spec
k8s.io/api/storage/v1.CSIDriverSpec {
        AttachRequired: *true,
        PodInfoOnMount: *false,
        VolumeLifecycleModes: []k8s.io/api/storage/v1.VolumeLifecycleMode len: 1, cap: 1, [
                "Persistent",
        ],
        StorageCapacity: *false,
        FSGroupPolicy: *"File",
        TokenRequests: []k8s.io/api/storage/v1.TokenRequest len: 0, cap: 0, nil,
        RequiresRepublish: *false,
        SELinuxMount: *bool nil,} <=====>
(dlv) p existingCsiDriver.Spec
k8s.io/api/storage/v1.CSIDriverSpec {
        AttachRequired: *true,
        PodInfoOnMount: *false,
        VolumeLifecycleModes: []k8s.io/api/storage/v1.VolumeLifecycleMode len: 1, cap: 1, [
                "Persistent",
        ],
        StorageCapacity: *false,
        FSGroupPolicy: *"File",
        TokenRequests: []k8s.io/api/storage/v1.TokenRequest len: 0, cap: 0, nil,
        RequiresRepublish: *false,
        SELinuxMount: *false,} <=====>
```
2. We need to mount csi configmap in Nodeplugin as well
3. Topology feature gate should be `false` by default
4. Fix typo in CSI plugin roles

Signed-off-by: Leela Venkaiah G <[email protected]>
@leelavg leelavg requested review from nb-ohad and Madhu-1 August 2, 2024 14:30
internal/utils/csi.go Show resolved Hide resolved
@Madhu-1 Madhu-1 merged commit 2f5495d into ceph:main Aug 2, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants