Installation on OpenStack stuck

[woland7]

Installation stuck

Bootstrap phase is compolet but then installation gets stucked at:

DEBUG Still waiting for the cluster to initialize: Multiple errors are preventing progress:
DEBUG * Could not update imagestream "openshift/driver-toolkit" (484 of 705): the server is down or not responding
DEBUG * Could not update oauthclient "console" (435 of 705): the server does not recognize this resource, check extension API servers
DEBUG * Could not update role "openshift-console-operator/prometheus-k8s" (640 of 705): resource may have been deleted

Version
4.8.0-0.okd-2021-11-14-052418

How reproducible
Okd 4.8 on openstack 16, 3 control plane nodes and 1 worker nodes

[vrutkovs]

Please attach log bundle

[woland7]

Ok so I re-run the installer and that error didn't appear but then the installer got stuck at this

DEBUG Still waiting for the cluster to initialize: Working towards 4.8.0-0.okd-2021-11-14-052418: 680 of 705 done (96% complete)
DEBUG Still waiting for the cluster to initialize: Some cluster operators are still updating: authentication, console, kube-apiserver

I also attached the bundle
bundle.zip

[woland7]

I was going through the event-filter.html and then I got the logs of the console pod saying this

error contacting auth provider (retrying in 10s): discovery through endpoint https://kubernetes.default.svc/.well-known/oauth-authorization-server failed: 404 Not Found

[woland7]

Basically that error seems caused by this other error

Message:               StaticPodsDegraded: pods "kube-apiserver-rcost-tqq9x-master-2" not found

StaticPodsDegraded: pods "kube-apiserver-rcost-tqq9x-master-0" not found

which I can get by running oc --kubeconfig=auth/kubeconfig describe clusteroperators.config.openshift.io kube-apiserver

In fact, inside the project default I found no pod deployed and yet there's a service, named kubernetes, whose endpoint is 10.0.2.150:6443

[woland7]

Ok false alarm, was experimenting with the flavors and brought the RAM for the control plane to 8GB, that implied a FailedScheduling for the kube-apiserver pods.

Yet I saw that at even at rest, the control planes reach20 GB RAM (with pagination), so I think I'll increase the RAM for the flavor beyond the min suggested value of 16 GB.