dns issues in OKD4.9/4.10(Failed unit NetworkManager-wait-online.service)

[jonasbartho]

Hi,

Provisioning/bootstrapping OKD4.9 or 4.10 from scratch on Openstack(UPI) goes pretty smooth.

Boot-times of the fedoracoreOS-nodes can however take up til 2 minutes or longer(which is way too long),and often we experience also slow or not working DNS resolving. I assume both issues are connected. (a plain fedorecoreOS(not OKD) boots under 30 seconds...)
We also see that "NetworkManager-wait-online.service" times out and gives a "failed" status after rebooting.

On the hosts(master/worker nodes) themselves we see that /etc/resolv.conf(managed by systemd-resolved) and /etc/coredns/Corefile are being used for resolving...

default /etc/resolv.conf after provisioning of the cluster:

nameserver 10.10.196.15(=ip that is automatically assigned to interface br-ex)
nameserver <company DNS-server gotten from Openstack subnet>
search local okd4.test.local.net

We assume that removing 10.10.196.15 can possibly fix our issue. The question now is if this is wise? If so, how to proceed? How can this be changed in an elegant way?

default /etc/coredns/Corefile after provisioning of the cluster:

. {
    errors
    bufsize 512
    health :18080
    forward . 127.0.0.53 {
        policy sequential
    }
    cache 30
    reload
    template IN A okd4.test.local.net {
        match .*.apps.okd4.test.local.net
        answer "{{ .Name }} 60 in {{ .Type }} 10.10.196.71"
        fallthrough
    }
    template IN AAAA okd4.test.local.net {
        match .*.apps.okd4.test.local.net
        fallthrough
    }
    template IN A okd4.test.local.net {
        match api.okd4.test.local.net
        answer "{{ .Name }} 60 in {{ .Type }} 10.10.196.69"
        fallthrough
    }
    template IN AAAA okd4.test.local.net {
        match api.okd4.test.local.net
        fallthrough
    }
    template IN A okd4.test.local.net {
        match api-int.okd4.test.local.net
        answer "{{ .Name }} 60 in {{ .Type }} 10.10.196.69"
        fallthrough
    }
    template IN AAAA okd4.test.local.net {
        match api-int.okd4.test.local.net
        fallthrough
    }
    hosts {
        10.10.196.84 controller0-osl2 controller0-osl2.okd4.test.local.net
        10.10.196.91 controller0-osl3 controller0-osl3.okd4.test.local.net
        10.10.196.72 controller0-osl4 controller0-osl4.okd4.test.local.net
        10.10.196.88 node0-osl2 node0-osl2.okd4.test.local.net
        10.10.196.68 node0-osl4 node0-osl4.okd4.test.local.net
        fallthrough
    }
}

Or can /etc/resolv.conf be ignored completely and do we just need to change forward . 127.0.0.53 in /etc/coredns/Corefile to our company's external DNS-resolver?

[vrutkovs]

forward . 127.0.0.53 {

It should not be a local address, see #978

[jonasbartho]

Right, thank you! Adding the suggested machineconfig-workaround in #978 solved it. :)