On minishift, admission controller webhook is not working #111
Comments
|
Looks like the "members" field for controller role is not getting mutated. |
|
Hi Similar results. I had to lower the memory request to 1Gi each as I am running it inside minishift on my laptop This is the config |
|
Different crash point this time, but I think it's a similar issue. The storageClass field is not being set to its default by the mutating webhook. We'll need to figure out why that webhook is not active. Tomorrow (Monday) we'll get together some debugging ideas. If the state of this particular KD installation has been changing, with some elements left in place and others updated (e.g. I see one indication in the log of KD starting up when a CDH cluster already exists), then it's possible that there is some out-of-sync-ness in the versions of those pieces that could be fixed by doing a teardown and clean re-deploy. However I suspect it's more likely that the webhook simply isn't being called for reasons that have to do with the particular k8s configuration. Quite possibly the default minishift configuration doesn't have that stuff active; we haven't tried this on minishift yet. |
joel-bluedata
added
Priority: High
Project: CR Validation
|
As an initial comment about that path: what we need to do is make sure that MutatingAdmissionWebhook is enabled. On OpenShift for example you have to do something like this (substituting MutatingAdmissionWebhook for ValidatingAdmissionWebhook): openshift/openshift-ansible#7983 (comment) minishift may also need some sort of action. From minishift/minishift#2676 I would guess it's not enabled by default. I haven't looked closely yet but there may be guidance on how to turn it on either in that issue thread or in one of these: |
|
You were right. I have activated the MutatingAdmissionWebhook and ValidatingAdmissionWebhook by enabling a minishift addon I killed the kubedirector pod but still getting similar errors |
|
The webhook is still not being called. If you want an easy way to for-sure test whether the webhook is working, you can go back to creating a cluster CR where one of the roles does not have an explicit member count. If the webhook is working, then as soon as you create the CR you should be able to read it back again (using kubectl or whatever) and see that a default value for the members property has been added to the CR. (Regardless of whether or not KD has then crashed.) @sandrich, did you restart the relevant services after changing the configuration? For example on OpenShift you would need to do I'm going to tag @tap in here briefly to see if he remembers any other relevant admission webhook shenanigans on OpenShift. BTW, looking ahead just a bit... this was not the only issue we hit on OpenShift, so I'm guessing there will be additional problems in minishift too unless things have changed since we last tried OpenShift. If you are wanting to go down this route you should also have a look at all of the other sub-issues of issue #1. If you don't mind giving some context: what's the situation that is leading you to use minishift for this? Convenience because you're used to it, or some other criterion? I ask because it might be hard for us to shake out all the issues here remotely -- it may require us to set up minishift ourselves and give it a go. However we don't have minishift support prioritized at the moment, so it's nice to gather info about whether we should re-prioritize. |
joel-bluedata
added
Project: Platform Support
|
I was looking at those OpenShift related issues again just now with @swamibluedata. issue #2: This is the webhook thing that we're talking about in the thread above. Perhaps this is just that the relevant controller services need to be restarted, perhaps it's something else. issue #3: I'm not sure what the symptoms of this RBAC issue were (@tap might remember, but he's at a conference right now). This may still need to be adjusted in your system if it's not something that minishift deals with more gracefully than "normal" OpenShift. issue #4: The symptom for this one was that the webhook wasn't able to bind to its port and KD would refuse to start. Since you don't seem to be suffering that problem, this one may not be an issue for you. issue #6: You'll probably need to make this change to the catalog entries. It looks like we may have a bit of schedule wiggle room here to give minishift a try ourselves in the near future and see how it pans out. |
joel-bluedata
changed the title
Hi @joel-bluedata thanks for looking into it. I will answer a bit later this week on the other issues and will have a closer look. |
Hi
I run minishift
Getting the following alerts in the kubedirector pod when I try to apply a cdh cluster
This is the custom config
The text was updated successfully, but these errors were encountered: