Replace Dex with external OIDC provider (Keycloak) in v1.9 and above

[jmmoreu]

Validation Checklist

• Is this a Kubeflow issue?
• Are you posting in the right repository ?
• Did you follow the installation guide https://github.com/kubeflow/manifests?tab=readme-ov-file ?
• Is the issue report properly structured and detailed with version numbers?
• Is this for Kubeflow development ?
• Would you like to work on this issue?
• You can join the CNCF Slack and access our meetings at the Kubeflow Community website. Our channel on the CNCF Slack is here #kubeflow-platform.

Version

1.9

Describe your issue

Version 1.9 introduces a major change replacing traditional oidc-authservice with oauth2-proxy. In fact, master's branch has already removed the oidc-authservice folder. Previous option allowed switching from Dex to Keycloak (or other oidc providers) by simply creating an overlay to modify params.env and secret_params.env (as shown in #2379 and #2513).

The corresponding kustomization.yaml removes oidc-authservice and installs oauth2-proxy with m2m-self-signed overlay. This uses oauth2-proxy-config.yaml which contains the cfg for traditional Dex service.

I assume that if we were to replace Dex with Keycloak we would need an overlay modifying this ConfigMap with the required fields. However, many of these fields in this cfg do not match exactly with previous fields to be changed in the previous approach (i.e params.env). Which fields should be changed and how?

Additionally, I assume that the secret_params CLIENT_ID and CLIENT_SECRET should be modified in this secretGenerator.

Steps to reproduce the issue

See above.

Put here any screenshots or videos (optional)

No response

[thesuperzapper]

@jmmoreu I want to note that Keycloak is a lot more complicated than Dex, and while it has more features, its purpose is really to run as your core authentication infrastructure, not to be the authentication infrastructure for a specific app.

Right now it's already possible to integrate default Kubeflow with a Keycloak deployment, by having Dex trust it.

What exactly are you wanting to achieve by this change?

[jmmoreu]

@thesuperzapper pretty much what was discussed in #2513 and #2379. I've been using Keycloak to manage users and groups access to third party apps including Kubeflow. I just want users to access kubeflow with their keycloak credentials.

Perhaps I am mistaken, but I believe that in my setting I am not using Dex at all. Also, in #2379 they really mean replacing Dex with Keycloak, right?

If what you mentioned about having dex trust my keycloak deployment works, that is ok to me. I would really appreciate some hints on how to achieve so, though. Thanks!

[thesuperzapper]

@jmmoreu Dex can connect to Keycloak through OpenID Connect, this is documented on the deployKF docs, which should give you context, even if you are not using deployKF:

• https://www.deploykf.org/guides/platform/deploykf-authentication/#keycloak

[kromanow94]

@jmmoreu so you're setup includes a k8s cluster running with self-signed certificates for the kube-apiserver and keycloak instead of dex? Or is keycloak also used for the kube-apiserver as well?

[jmmoreu]

@kromanow94 Yes, I have also configured the apiserver to perform OIDC authentication, following this tutorial. This is particularly useful to grant users access to the cluster, and attach VS code directly to the jupyter server pod, for instance.

I am fine with the approach @thesuperzapper suggested (I need to try it first, though). I just thought it would be possible to keep using my previous approach. If you think there's another way which might be interesting I'll be glad to hear it. Thanks!

[kromanow94]

I see. It's definitely possible to not use dex, although that would be an easier and more flexible option, for example if you also want to setup SSO for ArgoCD or another tool. You could configure all the services you deploy on your cluster (or wherever) to use dex directly and configure dex with a connector to any OIDC Provider.

But, if you're more interested with going directly with ouath2-proxy, this can be achieved by configuring the following field in the common/oauth2-proxy/base/oauth2-proxy-config.yaml:

oidc_issuer_url = "https://keycloak.example.com/auth/realms/master"

and removing those few as those are configured for in-cluster dex:

skip_auth_regex
skip_oidc_discovery
login_url
redeem_url
oidc_jwks_url

If you use the same OIDC Issuer for kube-apiserver and the oauth2-proxy, you can skip the EXTRA_JWT_ISSUERS.
You also don't have to use the m2m-self-signed overlay as the OIDC Issuer is served with publicly trusted certificates and go directly with m2m.

Also, you'd have to configure:

• ALLOW_SELF_SIGNED_ISSUER=false from common/oauth2-proxy/base/kustomization.yaml

• M2M_TOKEN_ISSUER=https://keycloak.example.com/auth/realms/master from common/oauth2-proxy/components/istio-m2m/kustomization.yaml

• Patch the common/oauth2-proxy/components/istio-external-auth to remove the requestauthentication.dex-jwt.yaml file and provide something like requestauthentication.keycloak-jwt.yaml that contains your issuer.

It makes sense to provide a documentation on how to do that. Also, I think the oauth2-proxy kustomization can be modified to provide more args directly through the env variables so there is no need to modify the oauth2-proxy Config Map, but there might be other challenges...

[jxdn]

I see. It's definitely possible to not use dex, although that would be an easier and more flexible option, for example if you also want to setup SSO for ArgoCD or another tool. You could configure all the services you deploy on your cluster (or wherever) to use dex directly and configure dex with a connector to any OIDC Provider.

But, if you're more interested with going directly with ouath2-proxy, this can be achieved by configuring the following field in the common/oauth2-proxy/base/oauth2-proxy-config.yaml:

oidc_issuer_url = "https://keycloak.example.com/auth/realms/master"

and removing those few as those are configured for in-cluster dex:

skip_auth_regex
skip_oidc_discovery
login_url
redeem_url
oidc_jwks_url

If you use the same OIDC Issuer for kube-apiserver and the oauth2-proxy, you can skip the EXTRA_JWT_ISSUERS. You also don't have to use the m2m-self-signed overlay as the OIDC Issuer is served with publicly trusted certificates and go directly with m2m.

Also, you'd have to configure:

• ALLOW_SELF_SIGNED_ISSUER=false from common/oauth2-proxy/base/kustomization.yaml
• M2M_TOKEN_ISSUER=https://keycloak.example.com/auth/realms/master from common/oauth2-proxy/components/istio-m2m/kustomization.yaml
• Patch the common/oauth2-proxy/components/istio-external-auth to remove the requestauthentication.dex-jwt.yaml file and provide something like requestauthentication.keycloak-jwt.yaml that contains your issuer.

It makes sense to provide a documentation on how to do that. Also, I think the oauth2-proxy kustomization can be modified to provide more args directly through the env variables so there is no need to modify the oauth2-proxy Config Map, but there might be other challenges...

HTTP/1.1 "" 302 12523 0.001 got this error , too many redirect.... need some guide on it

[jxdn]

I see. It's definitely possible to not use dex, although that would be an easier and more flexible option, for example if you also want to setup SSO for ArgoCD or another tool. You could configure all the services you deploy on your cluster (or wherever) to use dex directly and configure dex with a connector to any OIDC Provider.
But, if you're more interested with going directly with ouath2-proxy, this can be achieved by configuring the following field in the common/oauth2-proxy/base/oauth2-proxy-config.yaml:

oidc_issuer_url = "https://keycloak.example.com/auth/realms/master"

and removing those few as those are configured for in-cluster dex:

skip_auth_regex
skip_oidc_discovery
login_url
redeem_url
oidc_jwks_url

If you use the same OIDC Issuer for kube-apiserver and the oauth2-proxy, you can skip the EXTRA_JWT_ISSUERS. You also don't have to use the m2m-self-signed overlay as the OIDC Issuer is served with publicly trusted certificates and go directly with m2m.
Also, you'd have to configure:

• ALLOW_SELF_SIGNED_ISSUER=false from common/oauth2-proxy/base/kustomization.yaml
• M2M_TOKEN_ISSUER=https://keycloak.example.com/auth/realms/master from common/oauth2-proxy/components/istio-m2m/kustomization.yaml
• Patch the common/oauth2-proxy/components/istio-external-auth to remove the requestauthentication.dex-jwt.yaml file and provide something like requestauthentication.keycloak-jwt.yaml that contains your issuer.

It makes sense to provide a documentation on how to do that. Also, I think the oauth2-proxy kustomization can be modified to provide more args directly through the env variables so there is no need to modify the oauth2-proxy Config Map, but there might be other challenges...

HTTP/1.1 "" 302 12523 0.001 got this error , too many redirect.... need some guide on it

No valid authentication in request. Initiating login.

[juliusvonkohout]

Closing due to inactivity