bug: Problem setting clusterrole for using kubelet in the Kubernetes filter

[alidavodimehr]

Describe the issue

I am using Fluent Bit with the Fluent Operator Helm chart. I want to enable the kubelet feature in the Kubernetes filter, which requires modifying the clusterrole for Fluent Bit. I added the necessary permissions mentioned in the documentation to the RBAC rules section in the values file. However, after applying the helm chart, I am getting log errors.

2024-11-05T12:02:00Z  ERROR  Reconciler error  {"controller": "fluentbit", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "FluentBit", "FluentBit": {"name":"fluent-bit","namespace":"fluent-operator"}, "namespace": "fluent-operator", "name": "fluent-bit", "reconcileID": "b34c7734-3566-4392-a82f-7c21d2c7dac7", "error": "clusterroles.rbac.authorization.k8s.io \"fluent-operator-fluent-bit\" is forbidden: user \"system:serviceaccount:fluent-operator:fluent-operator\" (groups=[\"system:serviceaccounts\" \"system:serviceaccounts:fluent-operator\" \"system:authenticated\"]) is attempting to grant RBAC permissions not currently held:\n{APIGroups:[\"\"], Resources:[\"nodes\"], Verbs:[\"get\" \"list\" \"watch\"]}\n{APIGroups:[\"\"], Resources:[\"nodes/proxy\"], Verbs:[\"get\" \"list\" \"watch\"]}\n{APIGroups:[\"\"], Resources:[\"pods\"], Verbs:[\"list\" \"watch\"]}"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
  /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:324
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
  /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
  /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222

After investigation, I realized that the rules I want to assign to Fluent Bit must also be granted to the operator. After reviewing the Fluent Operator Helm chart, I found that the operator’s clusterrole is static, and there is no option to add rules through the Helm chart.

I can assist with modifying the Helm chart if needed.

To Reproduce

Apply the necessary changes mentioned in the documentation using the Helm chart

Expected behavior

Modify the Fluent Bit clusterrole to include the specified settings

Your Environment

- Fluent Operator version: 3.2.0
- Container Runtime: cri-o / kubernetes
- Operating system: Linux / Ubuntu
- Kernel version: 5.15.0-119 generic

How did you install fluent operator?

Helm Chart

Additional context

My values file

operator:
  resources:
    limits:
      cpu:
      memory:
    requests:
      cpu: 100m
      memory: 20Mi
  disableComponentControllers: "fluentd"

fluentbit:
  hostNetwork: true
  rbacRules:
    - apiGroups: [""]
      resources:
        - namespaces
        - pods
        - nodes
        - nodes/proxy
      verbs:
        - get
        - list
        - watch
  resources:
    limits:
      cpu:
      memory:
    requests:
      cpu: 10m
      memory: 25Mi
  input:
    tail:
      enable: false
    systemd:
      enable: false
    fluentBitMetrics:
      scrapeInterval: "15"
      scrapeOnStart: true
      tag: "fb.metrics"
  filter:
    kubernetes:
      enable: false
    containerd:
      enable: false
    systemd:
      enable: false
  output:
    prometheusMetricsExporter:
      match: "fb.metrics"
      metricsExporter:
        host: "0.0.0.0"
        port: 2020
        addLabels:
          app: "fluentbit"

fluentd:
  crdsEnable: false