Okta group impersonate limitation #2696

knopka · 2022-09-04T08:02:03Z

Describe the bug

Looks like this configuration https://github.com/weaveworks/weave-gitops/tree/main/charts/gitops-server#impersonate doesn't work

rbac:
  create: true
  impersonationResources: ["groups"]
  impersonationResourceNames: ["gitops-reader"]

I have configured login via an OIDC(Okta) provider. In Okta, group created and user added to this group. Okta Application and Authorization Server configured as well. Then I created ClusterRole and ClusterRoleBinding with subject as group name I created in Okta. Without rbac configuration block I mentioned above all works and I can see cluster data in UI, but after adding rbac configuration block UI is empty.

From Logs

2022-09-04T06:44:42.172Z	DEBUG	gitops.auth-server	auth/jwt.go:155	Found principal	{"user": "raman", "groups": ["wego-admin-rk"], "tokenLength": 0, "method": "*auth.JWTCookiePrincipalGetter"}
2022-09-04T06:44:42.172Z	DEBUG	gitops.auth-server	auth/jwt.go:155	Found principal	{"user": "raman", "groups": ["wego-admin-rk"], "tokenLength": 0, "method": "*auth.JWTCookiePrincipalGetter"}
2022-09-04T06:44:42.194Z	ERROR	gitops	clustersmngr/factory.go:367	failed filtering namespaces	{"cluster": "Default", "user": "raman", "error": "user namespace access: users \"raman\" is forbidden: User \"system:serviceaccount:flux-system:ww-gitops-weave-gitops\" cannot impersonate resource \"users\" in API group \"\" at the cluster scope"}

Environment

Weave-Gitops Version - helm chart version - 3.0.3
Flux Version - v0.31.2
Kubernetes version - v1.21.14-eks-18ef993

To Reproduce
Steps to reproduce the behavior:

Configure Okta (User, Group, Application and Authorization Server)
Create ClusterRole and ClusterRoleBinding with subject as group name created in Okta.
Add configuration below:

rbac:
  create: true
  impersonationResources: ["groups"]
  impersonationResourceNames: ["some-group"]

Expected behavior
After adding rbac configuration block impersonate limitation works and in UI we can see cluster data.

Actual Behavior
After adding rbac configuration block with specific group in UI there is no cluster data.

The text was updated successfully, but these errors were encountered:

LukaszRacon · 2022-09-15T03:19:58Z

Two issues:

impersonation for groups requires access to impersonate on users resource at cluster scope level (overall bad idea)
impersonationResourceNames: ["gitops-reader"] does not match group in logs "groups": ["wego-admin-rk"]

Following config requires access to impersonate on users resource at cluster scope level

rbac:
  create: true
  impersonationResources: ["groups"]
  impersonationResourceNames: ["gitops-reader"]

With current configuration it fails with error:
{"cluster": "Default", "user": "raman", "error": "user namespace access: users \"raman\" is forbidden: User \"system:serviceaccount:flux-system:ww-gitops-weave-gitops\" cannot impersonate resource \"users\" in API group \"\" at the cluster scope"}

Here is a fix (adjust names/namespaces accordingly)

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: ww-gitops-weave-gitops-patch
rules:
  - apiGroups:
      - ""
    resources:
      - users
    verbs:
      - impersonate
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: ww-gitops-weave-gitops-patch
subjects:
  - kind: ServiceAccount
    name: ww-gitops-weave-gitops
    namespace: flux-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ww-gitops-weave-gitops-patch

grglzrv · 2022-12-19T16:42:52Z

I have the same issue.
Configured Okta successfully with Weave server but not seeing any data.

Tried the aforementioned solutions but without success

i am getting this error

 2022-12-19T16:39:53.985Z    ERROR    gitops.core-server    server/version.go:105    error getting flux version    {"error": "error getting list of objects"}                                                                             │
│ 2022-12-19T16:39:55.200Z    INFO    gitops.auth-server    auth/jwt.go:77    attempt to read token from auth header                                                                                                                       │
│ 2022-12-19T16:39:55.201Z    INFO    gitops.auth-server    auth/jwt.go:77    attempt to read token from auth header

and "No data" into Weave UI

versions:
Kubernetes: v1.24.7-eks-fb459a0
Flux: 0.33
Weave GitOps:
v0.12.0

rbac config:

    rbac:
      create: true
      impersonationResourceNames: ["admin", "AdminGroup"]
      impersonationResources: ["users", "groups"]
      viewSecretsResourceNames: ["cluster-user-auth", "oidc-auth"]

cluster role and binding

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: wego-admin-cluster-role-okta
rules:
  - apiGroups: [""]
    resources: ["secrets", "pods" ]
    verbs: [ "get", "list" ]
  - apiGroups: ["apps"]
    resources: [ "deployments", "replicasets"]
    verbs: [ "get", "list" ]
  - apiGroups: ["kustomize.toolkit.fluxcd.io"]
    resources: [ "kustomizations" ]
    verbs: [ "get", "list", "patch" ]
  - apiGroups: ["helm.toolkit.fluxcd.io"]
    resources: [ "helmreleases" ]
    verbs: [ "get", "list", "patch" ]
  - apiGroups: ["source.toolkit.fluxcd.io"]
    resources: [ "buckets", "helmcharts", "gitrepositories", "helmrepositories", "ocirepositories" ]
    verbs: [ "get", "list", "patch" ]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["get", "watch", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: wego-cluster-admin-okta
subjects:
- kind: Group
  name: AdminGroup
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: wego-admin-cluster-role-okta
  apiGroup: rbac.authorization.k8s.io

also added and this "patch" cluster role and binding

  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRole
  metadata:
    name: flux-webui-weave-gitops-patch
  rules:
    - apiGroups:
        - ""
      resources:
        - users
        - groups
      verbs:
        - impersonate
  ---
  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRoleBinding
  metadata:
    name: flux-webui-weave-gitops-patch
  subjects:
    - kind: ServiceAccount
      name: flux-webui-weave-gitops
      namespace: flux-system
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: flux-webui-weave-gitops-patch

@JamWils do you know if anyone is working on it ?
@LukaszRacon did you tested it with K8s version 1.24?

Bare in mind that in k8s version the SA flux-webui-weave-gitops does not have token cuz the token has to be created manually

│ Name:                flux-webui-weave-gitops                                                                                                                                                                                             │
│ Namespace:           flux-system                                                                                                                                                                                                         │
│ Labels:              app.kubernetes.io/instance=flux-webui                                                                                                                                                                               │
│                      app.kubernetes.io/managed-by=Helm                                                                                                                                                                                   │
│                      app.kubernetes.io/name=weave-gitops                                                                                                                                                                                 │
│                      app.kubernetes.io/version=v0.12.0                                                                                                                                                                                   │
│                      helm.sh/chart=weave-gitops-4.0.9                                                                                                                                                                                    │
│                      helm.toolkit.fluxcd.io/name=flux-webui                                                                                                                                                                              │
│                      helm.toolkit.fluxcd.io/namespace=flux-system                                                                                                                                                                        │
│ Annotations:         meta.helm.sh/release-name: flux-webui                                                                                                                                                                               │
│                      meta.helm.sh/release-namespace: flux-system                                                                                                                                                                         │
│ Image pull secrets:  <none>                                                                                                                                                                                                              │
│ Mountable secrets:   <none>                                                                                                                                                                                                              │
│ Tokens:              flux-webui-weave-gitops-token                                                                                                                                                                                       │
│ Events:              <none>                                                                                                                                                                                                              │
│

for example:

---
apiVersion: v1
kind: Secret
metadata:
  name: flux-webui-weave-gitops-token 
  namespace: flux-system
  annotations:
    kubernetes.io/service-account.name: "flux-webui-weave-gitops"
type: kubernetes.io/flux-webui-weave-gitops-token

LukaszRacon · 2022-12-19T17:33:13Z

I am running a custom build that includes PR #2745 - I am using AzureAD with AKS 1.24.6

grglzrv · 2022-12-20T06:25:40Z

@JamWils do you know if there is any update from team/pesto on this matter?

mikalai-t · 2023-02-13T21:28:08Z

Hello. The same issue here - no data in UI when trying to restrict access based on groups in Azure (it shouldn't really matter as OAuth2/OIDC is more or less similarly implemented in any Identity Provider).
I had applied all the necessary fixes:

additional ClusterRole and ClusterRoleBinding to impersonate all the users - as suggested by @grglzrv

added cli not to request groups scope as AzureAD doesn't support it:

additionalArgs:
  - --auth-methods=oidc
  - --oidc-client-id={{ .clientID }}
  - --oidc-client-secret={{ .clientSecret }}
  - --oidc-redirect-url={{ .redirectURL }}
  - --oidc-issuer-url={{ .issuerURL }}
  - --custom-oidc-scopes=openid,profile,email

but ended up with the kind of filtering error when the app throws an error on the first group and stops processing the rest, e.g. my user belongs to 5 groups:

2023-02-13T20:44:05.587Z	DEBUG	gitops	middleware/middleware.go:57	request success	{"uri": "/v1/objects", "status": 200}
2023-02-13T20:44:06.213Z	DEBUG	gitops	middleware/middleware.go:57	request success	{"uri": "/", "status": 200}
2023-02-13T20:44:06.213Z	DEBUG	gitops	middleware/middleware.go:57	request success	{"uri": "/", "status": 200}
2023-02-13T20:44:10.690Z	INFO	gitops.auth-server	auth/jwt.go:77	attempt to read token from auth header
2023-02-13T20:44:10.690Z	DEBUG	gitops.auth-server	auth/jwt.go:54	parsing cookie JWT token	{"claimsConfig": {"Username":"email","Groups":"groups"}}
2023-02-13T20:44:10.690Z	DEBUG	gitops.auth-server	auth/jwt.go:161	Found principal	{"user": "[email protected]", "groups": ["1111-0000-2222-...", "3333-0000-4444-...", "5555-MyTarget-Group-UUID" ], "tokenLength": 0, "method": "*auth.JWTCookiePrincipalGetter"}
2023-02-13T20:44:10.988Z	INFO	gitops.auth-server	auth/jwt.go:77	attempt to read token from auth header
2023-02-13T20:44:10.988Z	DEBUG	gitops.auth-server	auth/jwt.go:54	parsing cookie JWT token	{"claimsConfig": {"Username":"email","Groups":"groups"}}
2023-02-13T20:44:10.989Z	DEBUG	gitops.auth-server	auth/jwt.go:161	Found principal ...omitted for brevity
2023-02-13T20:44:11.089Z	ERROR	gitops	clustersmngr/factory.go:566	failed filtering namespaces	{"cluster": "Default", "user": "[email protected]", "error": "user namespace access: groups \"1111-0000-2222-...\" is forbidden: User \"system:serviceaccount:flux-system:ww-gitops\" cannot impersonate resource \"groups\" in API group \"\" at the cluster scope"}

Also trying to filter by user throws exactly the same error (and a work around is to create another extra ClusterRole again), so I would say a way to build a final RBAC manifest from Helm chart could be improved.

aboutobza-talend · 2023-08-13T08:49:43Z

@grglzrv did you find any solution for the issue ?

grglzrv · 2023-08-13T09:10:21Z

@grglzrv did you find any solution for the issue ?

Unfortunately no, I gonna try again this week.

grglzrv · 2023-08-13T09:20:20Z

@aboutobza-talend you may try this HelmRelease - oidcSecret : https://github.com/FrankelJb/home-ops/blob/main/kubernetes/apps/flux-system/weave-gitops/helmrelease.yaml#L34-L39 and impersonationResourceNames: https://github.com/FrankelJb/home-ops/blob/main/kubernetes/apps/flux-system/weave-gitops/helmrelease.yaml#L60-L62

Cajga · 2023-09-19T12:31:31Z

additional ClusterRole and ClusterRoleBinding to impersonate all the users - as suggested by @grglzrv
added cli not to request groups scope as AzureAD doesn't support it:
additionalArgs:
  - --auth-methods=oidc
  - --oidc-client-id={{ .clientID }}
  - --oidc-client-secret={{ .clientSecret }}
  - --oidc-redirect-url={{ .redirectURL }}
  - --oidc-issuer-url={{ .issuerURL }}
  - --custom-oidc-scopes=openid,profile,email

AzureAD does support groups. Go to the app registration -> Token configuration -> Add groups claim

knopka added the bug Something isn't working label Sep 4, 2022

LukaszRacon mentioned this issue Sep 15, 2022

add support for Azure OIDC #2745

Closed

JamWils added the team/pesto label Sep 15, 2022

foot added the area/authentication label Feb 2, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Okta group impersonate limitation #2696

Okta group impersonate limitation #2696

knopka commented Sep 4, 2022

LukaszRacon commented Sep 15, 2022

grglzrv commented Dec 19, 2022 •

edited

Loading

LukaszRacon commented Dec 19, 2022

grglzrv commented Dec 20, 2022

mikalai-t commented Feb 13, 2023

aboutobza-talend commented Aug 13, 2023 •

edited

Loading

grglzrv commented Aug 13, 2023

grglzrv commented Aug 13, 2023 •

edited

Loading

Cajga commented Sep 19, 2023 •

edited

Loading

Okta group impersonate limitation #2696

Okta group impersonate limitation #2696

Comments

knopka commented Sep 4, 2022

LukaszRacon commented Sep 15, 2022

grglzrv commented Dec 19, 2022 • edited Loading

LukaszRacon commented Dec 19, 2022

grglzrv commented Dec 20, 2022

mikalai-t commented Feb 13, 2023

aboutobza-talend commented Aug 13, 2023 • edited Loading

grglzrv commented Aug 13, 2023

grglzrv commented Aug 13, 2023 • edited Loading

Cajga commented Sep 19, 2023 • edited Loading

grglzrv commented Dec 19, 2022 •

edited

Loading

aboutobza-talend commented Aug 13, 2023 •

edited

Loading

grglzrv commented Aug 13, 2023 •

edited

Loading

Cajga commented Sep 19, 2023 •

edited

Loading