From 88ebb4ba0cb3a6f44a2dc7a60e952b4f5b098441 Mon Sep 17 00:00:00 2001 From: Pranshu Srivastava Date: Tue, 5 Sep 2023 19:26:00 +0530 Subject: [PATCH] docs: add kube-rbac-proxy adds info about kube-rbac-proxy's high-level workflow --- .../Projects/Observability/kube-rbac-proxy.md | 106 ++++++++++++++++++ 1 file changed, 106 insertions(+) create mode 100644 content/Projects/Observability/kube-rbac-proxy.md diff --git a/content/Projects/Observability/kube-rbac-proxy.md b/content/Projects/Observability/kube-rbac-proxy.md new file mode 100644 index 0000000..e9b4a42 --- /dev/null +++ b/content/Projects/Observability/kube-rbac-proxy.md @@ -0,0 +1,106 @@ +# kube-rbac-proxy + +`kube-rbac-proxy`, as the name suggests, is a proxy that sits in front of a workload and performs authentication and authorization on its behalf, to make sure the requests passing through are the ones that it expects to. + +## Workflow + +The purpose of `kube-rbac-proxy` is to distinguish between calls made by same or different user(s) (or service account(s)) to endpoint(s) and protect them from unauthenticated or unauthorized resource access based on the Bearer tokens or the RBACs they hold, respectively. + +- [**Authentication**](https://github.com/rexagod/kube-rbac-proxy/blob/74181c75b8b6fbcde7eff1ca5fae353faac5cfae/pkg/authn/config.go#L33-L39): Both [OIDC JWT](https://github.com/rexagod/kube-rbac-proxy/blob/52e49fbdb75e009db4d02e3986e51fdba0526378/pkg/authn/oidc.go#L45-L63) and [delegated (header-based)](https://github.com/kubernetes/apiserver/blob/master/pkg/authentication/authenticatorfactory/delegating.go#L102-L112) authentication rely on Bearer tokens to authenticate the requests. The tokens are validated against the OIDC provider or the Kubernetes API server, respectively. Monitoring components use [mTLS](#downstream-usage) for client-server authentication, which provides are secure way to establish trust based on each other's digital certificates. The proxy can also possibly be configured to [skip the authentication step](https://github.com/rexagod/kube-rbac-proxy/blob/4c4a18fa7dd5d8069c0aa3a55028e06631621a52/pkg/authn/delegating.go#L53) altogether, in which case it will not validate the tokens and will assume that the requests are authenticated, which is not recommended. This token represents the identity of the user or service account that is making the request. A [`TokenReview` request is created](https://github.com/kubernetes/apiserver/blob/21bbcb57c672531fe8c431e1035405f9a4b061de/plugin/pkg/authenticator/token/webhook/webhook.go#L51-L53), and includes the bearer token that the client provided, which is then sent to the API server. The API server verifies the authenticity of the token and responds with the details of the authenticated party, if the operation was successful. + +- [**Authorization**](https://github.com/rexagod/kube-rbac-proxy/blob/1c7f88b5e951d25a493a175e93515068f5c77f3b/pkg/authz/auth.go#L31C1-L37): Once authentication is done, `kube-rbac-proxy` must then decide whether to allow the user's request to go through or not. A [`SubjectAccessReview` request is created](https://github.com/kubernetes/apiserver/blob/21bbcb57c672531fe8c431e1035405f9a4b061de/plugin/pkg/authorizer/webhook/webhook.go#L57-L59) for the API server, which allows for the review of the subject's access to a particular resource. Essentially, it checks whether the authenticated user or service account or has sufficient permissions to perform the desired action on the requested resource, based on the RBAC permissions granted to it. If so, the request is forwarded to the endpoint, otherwise it is rejected. Its worth mentioning that the HTTP verbs are internally mapped to their [corresponding RBAC verbs](https://github.com/rexagod/kube-rbac-proxy/blob/ccd5bc7fec36f9db0747033c2d698cc75a0e314c/pkg/proxy/proxy.go#L49-L60). + +- **Request/response forwarding and error handling**: Once the request is authenticated and authorized, it is forwarded to the endpoint. The response from the endpoint is then forwarded back to the client. If the request fails at any point, the proxy returns an error response to the client. If the authorization step fails, i.e., the client doesn't have the required permissions to access the requested resource, `kube-rbac-proxy` returns an [HTTP `403 Forbidden` status code](https://github.com/rexagod/kube-rbac-proxy/blob/9efde2a776fd516cfa082cc5f2c35c7f9e0e2689/pkg/filters/auth_test.go#L300) to the client and does not forward the request to the endpoint. In the case of a failed authentication, an [HTTP `401 Unauthorized` status code](https://github.com/rexagod/kube-rbac-proxy/blob/9efde2a776fd516cfa082cc5f2c35c7f9e0e2689/pkg/filters/auth_test.go#L290) is returned instead (note the distinction between *authentication* and *unauthorized* here). + +## Downstream usage + +- **Mutual TLS (mTLS or 2-way TLS)**: Client-server communications can also be secured using Mutual TLS (mTLS), which is a two-way authentication mechanism. Initially, the server provides its digital certificate to the client which validates the server's identity. The process is then reciprocated, as the client shares its digital certificate for authentication by the server. Following the successful completion of these authentication steps, a secure channel for encrypted communication is established, ensuring that data transfer between the entities is duly safeguarded. Below is a `Secret` that contains the server's certificate and key, which is then mounted as a volume in the`kube-rbac-proxy` pod. The `kube-rbac-proxy` container is configured to use the certificate and key for mTLS authentication. + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: kube-rbac-proxy +type: Opaque +data: + tls.crt: + tls.key: + ca.crt: +--- +apiVersion: apps/v1 +kind: Deployment +... +spec: + template: + spec: + containers: + - name: kube-rbac-proxy + image: quay.io/brancz/kube-rbac-proxy:v0.8.0 + args: + - "--tls-cert-file=/etc/tls/private/tls.crt" + - "--tls-private-key-file=/etc/tls/private/tls.key" + - "--client-ca-file=/etc/tls/client/client-ca.crt" + ... + volumeMounts: + - name: secret-volume + mountPath: /etc/tls + volumes: + - name: secret-volume + secret: + secretName: kube-rbac-proxy +``` + +CMO specifies the aforementioned CA certificate in the [metrics-client-ca](https://github.com/openshift/cluster-monitoring-operator/blob/6795f509e77cce6d24e5a3e371a432ca22e1a8e7/assets/cluster-monitoring-operator/metrics-client-ca.yaml) ConfigMap which is used to define client certificates for every `kube-rbac-proxy` container that's safeguarding a component. The component `Service` endpoints are secured using the generated TLS `Secret` annotating it with the `service.beta.openshift.io/serving-cert-secret-name`. Internally, this requests the `service-ca` controller to generate a `Secret` containing a certificate and key pair for the `${service.name}.${service.namespace}.svc`. These TLS manifests are then used in various component `ServiceMonitors` to define their TLS configurations, and within CMO to ensure a "mutual" acknowledgement between the two. + +- **Static authorization**: Static authorization involves configuring `kube-rbac-proxy` to allow access to certain resources or non-resources which are evaluated against the `Role` or `ClusterRole` RBAC permissions the user or the service account has. The example below demonstrates how this can be employed to give access to a known `ServiceAccount` to the `/metrics` endpoint. `/metrics` endpoints exposed by various monitoring components are protected this way. + +```yaml +apiVersion: v1 +kind: Secret +... +stringData: + # "path" is the path to match against the request path. + # "resourceRequest" is a boolean indicating whether the request is for a resource or not. + # "user" is the user to match against the request user. + # "verb" is the verb to match against the corresponding request RBAC verb. + config.yaml: |- + "authorization": + "static": + - "path": "/metrics" + "resourceRequest": false + "user": + "name": "system:serviceaccount:openshift-monitoring:prometheus-k8s" + "verb": "get" +``` + +On the other hand, the example below depicts restricted access to a resource, i.e., `monitoring.coreos.com/prometheusrules` in the `openshift-monitoring` namespace. Note that all applicable omitted configuration settings are interpreted as wildcards. + +```yaml +apiVersion: v1 +kind: Secret +... +stringData: + # "resourceAttributes" describes attributes available for resource request authorization. + # "rewrites" describes how SubjectAccessReview may be rewritten on a given request. + # "rewrites.byQueryParameter" describes which HTTP URL query parameter is to be used to rewrite a SubjectAccessReview + # on a given request. + config.yaml: |- + "authorization": + "resourceAttributes": + "apiGroup": "monitoring.coreos.com" + "namespace": "openshift-monitoring" + "resource": "prometheusrules" + "rewrites": + "byQueryParameter": + "name": "namespace" +``` + +For more details, refer to the `kube-rbac-proxy`'s [static authorization](https://github.com/rexagod/kube-rbac-proxy/blob/4a44b610cd12c4cfe076a2b306283d0598c1bb7a/examples/static-auth/README.md#L169) example. + +## Configuration + +Details on configuring `kube-rbac-proxy` under different scenarios can be found in the repository's [/examples](https://github.com/rexagod/kube-rbac-proxy/tree/9f436d46699dfd425f2682e4338069642b682892/examples) section. + +## Debugging + +In addition to enabling debug logs or compiling a custom binary with debugging capabilities (`-gcflags="all=-N -l"`), users can grep [the audit logs](https://docs.openshift.com/container-platform/4.13/security/audit-log-view.html) for more information about requests or responses concerning `kube-rbac-proxy`.