custom resources not found, please enable the respective API group #3620
Unanswered
yuqun20230302
asked this question in
Q&A
Replies: 1 comment
-
|
The Dex version needs to be updated for your Kubernetes, I'm afraid. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Preflight Checklist
Version
2.24.0
Storage Type
Kubernetes
Installation Type
Official container image
Expected Behavior
According to the 2.24.0 kubernetes.md document, the dex service is created normally
Actual Behavior
According to the 2.24.0 kubernetes.md document，An error occurred when the dex pod was started。
The error information is as follows：
time="2024-07-09T07:08:08Z" level=info msg="checking if custom resource authcodes.dex.coreos.com has been created already..."
time="2024-07-09T07:08:08Z" level=info msg="The custom resource authcodes.dex.coreos.com already available, skipping create"
time="2024-07-09T07:08:08Z" level=info msg="checking if custom resource authrequests.dex.coreos.com has been created already..."
time="2024-07-09T07:08:08Z" level=info msg="failed to list custom resource authrequests.dex.coreos.com, attempting to create: not found"
time="2024-07-09T07:08:08Z" level=error msg="custom resources not found, please enable the respective API group"
time="2024-07-09T07:08:08Z" level=info msg="checking if custom resource oauth2clients.dex.coreos.com has been created already..."
time="2024-07-09T07:08:08Z" level=info msg="failed to list custom resource oauth2clients.dex.coreos.com, attempting to create: not found"
time="2024-07-09T07:08:08Z" level=error msg="custom resources not found, please enable the respective API group"
time="2024-07-09T07:08:08Z" level=info msg="checking if custom resource signingkeies.dex.coreos.com has been created already..."
time="2024-07-09T07:08:08Z" level=info msg="failed to list custom resource signingkeies.dex.coreos.com, attempting to create: not found"
time="2024-07-09T07:08:08Z" level=error msg="custom resources not found, please enable the respective API group"
Steps To Reproduce
1.Configuring the OpenID Connect plugin
vim /usr/lib/systemd/system/kube-apiserver.service ExecStart=/usr/local/bin/kube-apiserver \ --v=2 \ --allow-privileged=true \ --bind-address=0.0.0.0 \ --secure-port=6443 \ --advertise-address=172.16.58.101 \ --service-cluster-ip-range=192.168.0.0/16 \ --service-node-port-range=30000-32767 \ --etcd-servers=https://172.16.58.101:2379,https://172.16.58.102:2379,https://172.16.58.103:2379 \ --etcd-cafile=/etc/etcd/ssl/etcd-ca.pem \ --etcd-certfile=/etc/etcd/ssl/etcd.pem \ --etcd-keyfile=/etc/etcd/ssl/etcd-key.pem \ --client-ca-file=/etc/kubernetes/pki/ca.pem \ --tls-cert-file=/etc/kubernetes/pki/apiserver.pem \ --tls-private-key-file=/etc/kubernetes/pki/apiserver-key.pem \ --kubelet-client-certificate=/etc/kubernetes/pki/apiserver.pem \ --kubelet-client-key=/etc/kubernetes/pki/apiserver-key.pem \ --service-account-key-file=/etc/kubernetes/pki/sa.pub \ --service-account-signing-key-file=/etc/kubernetes/pki/sa.key \ --service-account-issuer=https://kubernetes.default.svc.cluster.local \ --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname \ --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,ResourceQuota \ --authorization-mode=Node,RBAC \ --enable-bootstrap-token-auth=true \ --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.pem \ --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.pem \ --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client-key.pem \ --requestheader-allowed-names=front-proxy-client \ --requestheader-group-headers=X-Remote-Group \ --requestheader-extra-headers-prefix=X-Remote-Extra- \ --requestheader-username-headers=X-Remote-User \ --tls-cipher-suites=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA # --token-auth-file=/etc/kubernetes/token.csv --oidc-issuer-url=https://dex.example.com:32000 \ --oidc-client-id=example-app \ --oidc-ca-file=/root/yq/dex/ssl/ca.pem \ --oidc-username-claim=email \ --oidc-groups-claim=groups systemctl daemon-reload && systemctl restart --now kube-apiserver2.Deploying dex on Kubernetes
1).Generate TLS assets
2).Configure the API server
3). crds
kubectl apply -f dex-crd.yaml apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: name: authcodes.dex.coreos.com spec: group: dex.coreos.com names: kind: AuthCode listKind: AuthCodeList plural: authcodes singular: authcode scope: Namespaced versions: - name: v1 served: true storage: true schema: openAPIV3Schema: type: object properties: spec: type: object4).Deploy the Dex server
kubectl apply -f dex.yaml -n auth apiVersion: apps/v1 kind: Deployment metadata: labels: app: dex name: dex spec: replicas: 1 selector: matchLabels: app: dex template: metadata: labels: app: dex spec: serviceAccountName: dex # This is created below containers: - image: 172.16.58.104:28081/kubeflow/dex:v2.24.0 name: dex command: ["/usr/local/bin/dex", "serve", "/etc/dex/cfg/config.yaml"] ports: - name: https containerPort: 5556 volumeMounts: - name: config mountPath: /etc/dex/cfg - name: tls mountPath: /etc/dex/tls env: - name: KUBERNETES_POD_NAMESPACE value: auth - name: GITHUB_CLIENT_ID valueFrom: secretKeyRef: name: github-client key: client-id - name: GITHUB_CLIENT_SECRET valueFrom: secretKeyRef: name: github-client key: client-secret volumes: - name: config configMap: name: dex items: - key: config.yaml path: config.yaml - name: tls secret: secretName: dex.example.com.tls --- kind: ConfigMap apiVersion: v1 metadata: name: dex data: config.yaml: | issuer: https://dex.example.com:32000 storage: type: kubernetes config: inCluster: true web: https: 0.0.0.0:5556 tlsCert: /etc/dex/tls/tls.crt tlsKey: /etc/dex/tls/tls.key connectors: - type: github id: github name: GitHub config: clientID: $GITHUB_CLIENT_ID clientSecret: $GITHUB_CLIENT_SECRET redirectURI: https://dex.example.com:32000/callback org: kubernetes oauth2: skipApprovalScreen: true staticClients: - id: example-app redirectURIs: - 'http://127.0.0.1:5555/callback' name: 'Example App' secret: ZXhhbXBsZS1hcHAtc2VjcmV0 enablePasswordDB: true staticPasswords: - email: "[email protected]" # bcrypt hash of the string "password" hash: "$2a$10$2b2cU8CPhOTaGrs1HRQuAueS7JTT5ZHsHSzYiFPm1leZck7Mc8T4W" username: "admin" userID: "08a8684b-db88-4b73-90a9-3cd1661f5466" --- apiVersion: v1 kind: Service metadata: name: dex spec: type: NodePort ports: - name: dex port: 5556 protocol: TCP targetPort: 5556 nodePort: 32000 selector: app: dex --- apiVersion: v1 kind: ServiceAccount metadata: labels: app: dex name: dex --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: dex rules: - apiGroups: ["dex.coreos.com"] # API group created by dex resources: ["*"] verbs: ["*"] - apiGroups: ["apiextensions.k8s.io"] resources: ["customresourcedefinitions"] verbs: ["create"] # To manage its own resources, dex must be able to create customresourcedefinitions --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: dex roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: dex subjects: - kind: ServiceAccount name: dex # Service account assigned to the dex pod, created above namespace: auth # The namespace dex is running inAdditional Information
kubectl version Client Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.6", GitCommit:"741c8db18a52787d734cbe4795f0b4ad860906d6", GitTreeState:"clean", BuildDate:"2023-09-13T09:21:34Z", GoVersion:"go1.20.8", Compiler:"gc", Platform:"linux/amd64"} Kustomize Version: v5.0.1 Server Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.6", GitCommit:"741c8db18a52787d734cbe4795f0b4ad860906d6", GitTreeState:"clean", BuildDate:"2023-09-13T09:14:09Z", GoVersion:"go1.20.8", Compiler:"gc", Platform:"linux/amd64"}Configuration
No response
Logs
Beta Was this translation helpful? Give feedback.
All reactions