improved logging/output options

[strajansebastian]

Hello,

I get the following error/warning message when running the job at the bottom in a rancher cluster on a worker node (on master I don't get this and the job is similar)

Output

AuditEnv not explicit for check (2.1), where bin path cannot be determined
AuditEnv not explicit for check (2.2), where bin path cannot be determined
AuditEnv not explicit for check (2.3), where bin path cannot be determined
AuditEnv not explicit for check (2.4), where bin path cannot be determined
AuditEnv not explicit for check (2.5), where bin path cannot be determined
AuditEnv not explicit for check (2.6), where bin path cannot be determined
AuditEnv not explicit for check (2.7), where bin path cannot be determined
... json printed lines removed ...

Found that the lines from above are from here: https://github.com/aquasecurity/kube-bench/blob/main/cmd/common.go#L136
From what I could find out fmt in go lang does support fmt.SetOutput to redirect to a file for example https://stackoverflow.com/questions/55149264/how-to-log-fmt-printf-in-custom-file.

I'm doing a decode of the kube-bench run output in json in python, but it fails due to the AuditEnv err which is not in the expected format.
Haven't tried yet --outputfile which should redirect the json to a file, which I can try to read on a later date... but this complicates a bit the whole process...

Do you know if any other options of logging combinations from kube-bench that are able to suppress the lines from above (or any other print message that isn't in json) without having to create a custom filter to suppress this type of output?
Also, not sure if something similar to https://docs.python.org/3/howto/logging.html can be done in golang to implement a more configurable log output. For example in python you can specify stderr/stdout, a file or the level of verbosity and if logging is specified as an option to the script you can have a lot of output possible combinations based on your used options.

---
apiVersion: batch/v1
kind: Job
metadata:
  name: kube-bench-worker
spec:
  template:
    metadata:
      labels:
        app: kube-bench-worker
    spec:
      hostPID: true
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: "kubernetes.io/os"
                operator: In
                values:
                - "linux"
      containers:
        - name: kube-bench
          image: harbor.yardiapp.com/dockerhub/aquasec/kube-bench:v0.6.7
          command: ["kube-bench", "run", "--json"]
          volumeMounts:
            - name: var-lib-etcd
              mountPath: /var/lib/etcd
              readOnly: true
            - name: var-lib-kubelet
              mountPath: /var/lib/kubelet
              readOnly: true
            - name: var-lib-kube-scheduler
              mountPath: /var/lib/kube-scheduler
              readOnly: true
            - name: var-lib-kube-controller-manager
              mountPath: /var/lib/kube-controller-manager
              readOnly: true
            - name: etc-systemd
              mountPath: /etc/systemd
              readOnly: true
            - name: lib-systemd
              mountPath: /lib/systemd/
              readOnly: true
            - name: srv-kubernetes
              mountPath: /srv/kubernetes/
              readOnly: true
            - name: etc-kubernetes
              mountPath: /etc/kubernetes
              readOnly: true
              # /usr/local/mount-from-host/bin is mounted to access kubectl / kubelet, for auto-detecting the Kubernetes version.
              # You can omit this mount if you specify --version as part of the command.
            - name: usr-bin
              mountPath: /usr/local/mount-from-host/bin
              readOnly: true
            - name: etc-cni-netd
              mountPath: /etc/cni/net.d/
              readOnly: true
            - name: opt-cni-bin
              mountPath: /opt/cni/bin/
              readOnly: true
      restartPolicy: Never
      volumes:
        - name: var-lib-etcd
          hostPath:
            path: "/var/lib/etcd"
        - name: var-lib-kubelet
          hostPath:
            path: "/var/lib/kubelet"
        - name: var-lib-kube-scheduler
          hostPath:
            path: "/var/lib/kube-scheduler"
        - name: var-lib-kube-controller-manager
          hostPath:
            path: "/var/lib/kube-controller-manager"
        - name: etc-systemd
          hostPath:
            path: "/etc/systemd"
        - name: lib-systemd
          hostPath:
            path: "/lib/systemd"
        - name: srv-kubernetes
          hostPath:
            path: "/srv/kubernetes"
        - name: etc-kubernetes
          hostPath:
            path: "/etc/kubernetes"
        - name: usr-bin
          hostPath:
            path: "/usr/bin"
        - name: etc-cni-netd
          hostPath:
            path: "/etc/cni/net.d/"
        - name: opt-cni-bin
          hostPath:
            path: "/opt/cni/bin/"
      tolerations:
      - effect: "NoSchedule"
        key: "type"
        operator: "Exists"
      - effect: "NoSchedule"
        key: "cattle.io/os"
        operator: "Equal"
        value: "linux"

Thanks,
Sebastian

[mozillazg]

Found that the lines from above are from here: https://github.com/aquasecurity/kube-bench/blob/main/cmd/common.go#L136

Thanks for reporting. This code can be improved.

Do you know if any other options of logging combinations from kube-bench that are able to suppress the lines from above (or any other print message that isn't in json) without having to create a custom filter to suppress this type of output?

There is no way to do it. Run kube-bench with the --outputfile flag is recommended way for this case.

Also, not sure if something similar to https://docs.python.org/3/howto/logging.html can be done in golang to implement a more configurable log output. For example in python you can specify stderr/stdout, a file or the level of verbosity and if logging is specified as an option to the script you can have a lot of output possible combinations based on your used options.

You can see some flags for control log output via run kube-bench -h |grep log , but it's not as powerful as the logging module.