Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suggesting more restrictive k8s RBAC and PSP #662

Open
bygui86 opened this issue Dec 18, 2020 · 1 comment
Open

Suggesting more restrictive k8s RBAC and PSP #662

bygui86 opened this issue Dec 18, 2020 · 1 comment
Labels
enhancement New feature or request help wanted Extra attention is needed pinned priority-low question Further information is requested
Milestone

Comments

@bygui86
Copy link

bygui86 commented Dec 18, 2020

Describe the bug:
I think that RBAC and PSP applied to logging-operator, fluent-bit and fluentd are too permissive.

Proposal:
I propose following RBAC and PSP to restrict as much as possible permissions to components, without jeopardizing or influencing normal functionalities:

operator RBAC + PSP:

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: logging-restricted
  annotations:
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: runtime/default
    seccomp.security.alpha.kubernetes.io/defaultProfileName:  runtime/default
    apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default
    apparmor.security.beta.kubernetes.io/defaultProfileName:  runtime/default
spec:
  privileged: false
  allowPrivilegeEscalation: false
  requiredDropCapabilities:
    - ALL
  volumes:
    - configMap
    - secret
    - persistentVolumeClaim
    - emptyDir
    - projected
    - downwardAPI
  hostPID: false
  hostIPC: false
  hostNetwork: false
  runAsUser:
    rule: MustRunAsNonRoot
  runAsGroup:
    rule: MustRunAs
    ranges:
      - min: 100
        max: 65535
  supplementalGroups:
    rule: MustRunAs
    ranges:
      - min: 1
        max: 65535
  fsGroup:
    rule: MustRunAs
    ranges:
      - min: 1
        max: 65535
  seLinux:
    rule: RunAsAny
  readOnlyRootFilesystem: true
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: logging-operator
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: logging-operator
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: logging-operator
subjects:
  - kind: ServiceAccount
    name: logging-operator
    namespace: logging
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: logging-operator
rules:
  - apiGroups:
      - policy
    resources:
      - podsecuritypolicies
    verbs:
      - use
    resourceNames:
      - logging-restricted
  - apiGroups:
      - batch
    resources:
      - jobs
    verbs:
      - "*"
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: logging-operator
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: logging-operator
subjects:
  - kind: ServiceAccount
    name: logging-operator
    namespace: logging
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: logging-operator
rules:
  - apiGroups:
      - policy
    resources:
      - podsecuritypolicies
    verbs:
      - use
    resourceNames:
      - logging-restricted
  - apiGroups:
      - monitoring.coreos.com
    resources:
      - servicemonitors
      - podmonitors
      - prometheusrules
    verbs:
      - "*"
  - apiGroups:
      - apiextensions.k8s.io
    resources:
      - customresourcedefinitions
    verbs:
      - list
      - watch
  - apiGroups:
      - logging.banzaicloud.io
    resources:
      - "*"
    verbs:
      - "*"
  - apiGroups:
      - ""
    resources:
      - nodes
      - namespaces
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - pods
      - services
      - endpoints
      - configmaps
      - secrets
      - persistentvolumeclaims
    verbs:
      - "*"
  - apiGroups:
      - apps
    resources:
      - deployments
      - daemonsets
      - statefulsets
    verbs:
      - "*"
  - apiGroups:
      - ""
      - rbac.authorization.k8s.io
    resources:
      - serviceaccounts
      - roles
      - rolebindings
      - clusterroles
      - clusterrolebindings
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - policy
    resources:
      - podsecuritypolicies
    verbs:
      - get
      - list
      - watch

fluent-bit RBAC + PSP:

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: logging-fluentbit-restricted
  annotations:
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: runtime/default
    seccomp.security.alpha.kubernetes.io/defaultProfileName:  runtime/default
    apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default
    apparmor.security.beta.kubernetes.io/defaultProfileName:  runtime/default
spec:
  privileged: false
  allowPrivilegeEscalation: false
  requiredDropCapabilities:
    - ALL
  hostPID: false
  hostIPC: false
  hostNetwork: false
  runAsGroup:
    rule: MustRunAs
    ranges:
      - min: 100
        max: 65535
  supplementalGroups:
    rule: MustRunAs
    ranges:
      - min: 1
        max: 65535
  fsGroup:
    rule: MustRunAs
    ranges:
      - min: 1
        max: 65535
  seLinux:
    rule: RunAsAny
  readOnlyRootFilesystem: true
  # DIFFERENCES from logging-restricted
  volumes:
    - configMap
    - secret
    - emptyDir
    - hostPath
  allowedHostPaths:
    # why this folder even considering it's empty? anyway read only...
    - pathPrefix: /var/lib/docker/containers
      readOnly: true
    - pathPrefix: /var/log
      readOnly: false
  runAsUser:
    rule: RunAsAny
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: logging-fluentbit
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: logging-fluentbit
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: logging-fluentbit
subjects:
- kind: ServiceAccount
  name: logging-fluentbit
  namespace: logging
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: logging-fluentbit
rules:
  - apiGroups:
      - policy
    resources:
      - podsecuritypolicies
    verbs:
      - use
    resourceNames:
      - logging-fluentbit-restricted
  - apiGroups:
    - ""
    resources:
    - pods
    - namespaces
    verbs:
    - get
    - list
    - watch

fluentd RBAC + PSP:

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: logging-fluentd-restricted
  annotations:
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: runtime/default
    seccomp.security.alpha.kubernetes.io/defaultProfileName:  runtime/default
    apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default
    apparmor.security.beta.kubernetes.io/defaultProfileName:  runtime/default
spec:
  privileged: false
  allowPrivilegeEscalation: false
  requiredDropCapabilities:
    - ALL
  hostPID: false
  hostIPC: false
  hostNetwork: false
  runAsUser:
    rule: MustRunAsNonRoot
  runAsGroup:
    rule: MustRunAs
    ranges:
      - min: 100
        max: 65535
  supplementalGroups:
    rule: MustRunAs
    ranges:
      - min: 1
        max: 65535
  fsGroup:
    rule: MustRunAs
    ranges:
      - min: 1
        max: 65535
  seLinux:
    rule: RunAsAny
  # DIFFERENCES from logging-restricted
  volumes:
    - configMap
    - secret
    - persistentVolumeClaim
    - emptyDir
    - hostPath
  # readOnlyRootFilesystem: true
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: logging-fluentd
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: logging-fluentd
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: logging-fluentd
subjects:
  - kind: ServiceAccount
    name: logging-fluentd
    namespace: logging
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: logging-fluentd
rules:
  - apiGroups:
      - policy
    resources:
      - podsecuritypolicies
    verbs:
      - use
    resourceNames:
      - logging-fluentd-restricted
  - apiGroups:
      - ""
    resources:
      - configmaps
      - secrets
    verbs:
      - "*"

logging usage example:

apiVersion: logging.banzaicloud.io/v1beta1
kind: Logging
metadata:
  name: logging-components
spec:
  enableRecreateWorkloadOnImmutableFieldChange: true
  controlNamespace: logging
  fluentbit:
    metrics:
      serviceMonitor: true
    tolerations:
      - key: node-role.kubernetes.io/master
        operator: Exists
        effect: NoSchedule
      - operator: Exists
        effect: NoExecute
      - operator: Exists
        effect: NoSchedule
    security:
      roleBasedAccessControlCreate: false
      podSecurityPolicyCreate: false
      serviceAccount: logging-fluentbit
      podSecurityContext:
        fsGroup: 101
  fluentd:
    scaling:
      replicas: 3
    metrics:
      serviceMonitor: true
    livenessDefaultCheck: true
    security:
      roleBasedAccessControlCreate: false
      podSecurityPolicyCreate: false
      serviceAccount: logging-fluentd
      securityContext:
        readOnlyRootFilesystem: false
      podSecurityContext:
        runAsUser: 100
        runAsGroup: 100
        fsGroup: 101

Environment details:
Above proposal were tested on an environment with following details:

  • Kubernetes version: v1.17.14
  • Cloud-provider/provisioner: GKE
  • logging-operator version: 3.8.3
  • Install method: static manifests

Looking forward your feedback!!

/kind bug

@bygui86 bygui86 added the bug Something isn't working label Dec 18, 2020
@stale stale bot added the wontfix This will not be worked on label Apr 12, 2023
@pepov pepov added enhancement New feature or request help wanted Extra attention is needed and removed wontfix This will not be worked on bug Something isn't working labels Apr 17, 2023
@stale stale bot added the wontfix This will not be worked on label Jun 16, 2023
@aslafy-z aslafy-z removed the wontfix This will not be worked on label Jun 16, 2023
@pepov pepov added this to the 5.0 milestone Jul 6, 2023
@pepov pepov modified the milestones: 5.0, 4.x Aug 8, 2023
@stale stale bot added the wontfix This will not be worked on label Oct 7, 2023
@aslafy-z aslafy-z added the pinned label Oct 7, 2023
@stale stale bot removed the wontfix This will not be worked on label Oct 7, 2023
@pepov pepov removed the help wanted Extra attention is needed label Jun 18, 2024
@pepov
Copy link
Member

pepov commented Jun 18, 2024

removed the help wanted label as:

  • psp is not supported anymore in the chart
  • functionality and rbac will be radically simplified in a future version

@kube-logging kube-logging deleted a comment from stale bot Nov 27, 2024
@kube-logging kube-logging deleted a comment from stale bot Nov 27, 2024
@kube-logging kube-logging deleted a comment from stale bot Nov 27, 2024
@csatib02 csatib02 added help wanted Extra attention is needed question Further information is requested labels Nov 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed pinned priority-low question Further information is requested
Projects
None yet
Development

No branches or pull requests

4 participants