-
Notifications
You must be signed in to change notification settings - Fork 68
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multicluster tutorial #681
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice tutorial, @leandroberetta !
It is very detailed and and think it will be very useful.
I had added some comments, some are just suggestions.
@@ -0,0 +1,9 @@ | |||
--- | |||
title: Travels Demo, Now Multicluster | |||
description: Learn how to configure and use Kiali in a Istio multicluster scenario. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
in an Istio
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we can link here? https://deploy-preview-681--kiali.netlify.app/docs/configuration/multi-cluster/ I think is useful to have related documentation linked, in case the reader needs more contexts about multi cluster.
Also, sometimes is confusing for me, how to write the Multi cluster word :) But as in the related link is written as multi-cluster, it would make sense to be consistent also here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mentioning the docs is something important, yes, I will add that.
Regarding to the multicluster word, I saw that in Istio, they called "multicluster", but I can change it, for the sake of consistency.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think "multicluster" is fine - especially if that's how Istio uses it. It definitely isn't two words (not "multi cluster") but I have seen it with a hypen ("multi-cluster"). I could go either way - either one word or with the hyphen (not two words though - "multi" isn't really a word in English).
istioctl install -y --set values.pilot.env.EXTERNAL_ISTIOD=true --context=$CLUSTER_EAST -f $CLUSTER_EAST.yaml | ||
``` | ||
|
||
After the installation, we need to create what we called an “east-west” gateway. It’s an ingress gateway just for the cross cluster configuration as we are opting to use the installation for different networks (this will be the case in the majority of the productive scenarios). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
production scenarios?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah probably :)
kubectl config use-context $CLUSTER_EAST | ||
|
||
helm upgrade --install --namespace istio-system --set kubernetes_config.cache_enabled=false --set auth.strategy=anonymous --set deployment.logger.log_level=debug --set deployment.ingress.enabled=true --repo https://kiali.org/helm-charts kiali-server kiali-server | ||
``` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a suggestion, maybe we can also say how to verify that Kiali is running?
Something like accessing Kiali in http://..
weight: 8 | ||
--- | ||
|
||
This installation will be different as this cluster will be a remote: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe it is possible to explain a bit why is different? Like the control plane will not be installed, just the CRDs, and it will be managed by a external CP?
Finally, we create the east-west gateway | ||
|
||
``` | ||
samples/multicluster/gen-eastwest-gateway.sh \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't be?
$ISTIO_DIR/samples/...
Finally, upgrade the installation for Kiali to pick up the secret: | ||
|
||
``` | ||
helm upgrade \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had some issues in this line:
Error: "helm upgrade" requires 2 arguments
Probably an indentation error, but if I put this in the same line:
Error: UPGRADE FAILED: "kiali-server" has no deployed releases
I guess that should be in istio-system
namespace, in this case I get:
Error: UPGRADE FAILED: cannot patch "kiali" with kind ClusterRoleBinding: ClusterRoleBinding.rbac.authorization.k8s.io "kiali" is invalid: roleRef: Invalid value: rbac.RoleRef{APIGroup:"rbac.authorization.k8s.io", Kind:"ClusterRole", Name:"kiali-viewer"}: cannot change roleRef
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will check this, it's rare because I tested this but I will double check.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
overall, very nice. worked all the way through (well, except I needed to pass in --driver kvm2
when starting minikube to avoid getting some startup warnings, but that could be an issue with my own setup)
content/en/docs/Tutorials/multicluster/04-Install-Istio-east-cluster.md
Outdated
Show resolved
Hide resolved
content/en/docs/Tutorials/multicluster/06-Install-Travels-east-cluster.md
Outdated
Show resolved
Hide resolved
content/en/docs/Tutorials/multicluster/06-Install-Travels-east-cluster.md
Outdated
Show resolved
Hide resolved
content/en/docs/Tutorials/multicluster/06-Install-Travels-east-cluster.md
Outdated
Show resolved
Hide resolved
content/en/docs/Tutorials/multicluster/08-Install-Istio-west-cluster.md
Outdated
Show resolved
Hide resolved
content/en/docs/Tutorials/multicluster/08-Install-Istio-west-cluster.md
Outdated
Show resolved
Hide resolved
content/en/docs/Tutorials/multicluster/10-Install-Travels-west-cluster copy.md
Outdated
Show resolved
Hide resolved
…luster.md Co-authored-by: John Mazzitelli <[email protected]>
Co-authored-by: John Mazzitelli <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The tutorial is great, thanks for putting all this together. Just a few suggestions/questions.
@@ -0,0 +1,10 @@ | |||
--- | |||
title: Travels Demo, Now Multicluster |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe remove the Now
part and just have this be:
title: Travels Demo, Now Multicluster | |
title: Travels Demo Multicluster |
Users may not do the tutorial sequentially or they might skip the single cluster tutorial altogether.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think you need to go into so much detail on the primary-remote deployment model. I think it's enough to link to the istio docs and then provide more detail here on the setup in this tutorial i.e. primary-remote, multiple networks, federated prometheus, anonymous auth.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe it would also be helpful to have somewhere a "jump to installing Kiali" section if you are trying to run the tutorial on an existing primary-remote deployment and don't want to go through all the minikube setup.
* Istio v1.18.1 | ||
* Kiali v1.70 | ||
|
||
Clusters are provided by minikube instances, but we can choose others instead, like OpenShift or just vanilla Kubernetes installations. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Clusters are provided by minikube instances, but we can choose others instead, like OpenShift or just vanilla Kubernetes installations. | |
Clusters are provided by minikube instances, but this tutorial should work on on any Kubernetes environment. |
CLUSTER_EAST="east" | ||
CLUSTER_WEST="west" | ||
ISTIO_DIR="absolute-path-to-istio-folder" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Assuming a bash shell, do these need export
in front of them in order to be set for the following commands?
network: network1 | ||
EOF | ||
|
||
istioctl install -y --set values.pilot.env.EXTERNAL_ISTIOD=true --context=$CLUSTER_EAST -f $CLUSTER_EAST.yaml |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This fails if you don't have istioctl in your PATH. I think $ISTIO_DIR/bin/istioctl
would work though? It's also fine if we expect istioctl to be in your PATH.
|
||
So far, we installed everything on one cluster, similarly to the Travels tutorial for a single cluster. | ||
|
||
Now we will expand this topology to include a remote cluster. As we commented this situation can be very common in a production scenario, either because we might want to split some applications into different clusters, generally because they are maintained by different developers or for high availability or just making applications available in other zones to reduce latencies. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As with the intro, I'm not sure we need to include motivations for adopting primary-remote here in the tutorial.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's ok to give people some sort of use cases, but I'm ok if you prefer to remove it. I'm working on a blog post that might include thise motivations.
Run the following commands to deploy the second cluster: | ||
|
||
``` | ||
minikube start -p $CLUSTER_WEST --network istio --memory 8g --cpus 4 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I get this error when running with the docker driver (the default when you don't provide one):
Startup with docker driver failed, trying with alternate driver kvm2: Failed to start host: can't create with that IP, address already in use
it then switches to kvm2 because I have that installed on my system but if I didn't the tutorial might fail at that point altogether.
|
||
## Prometheus federation | ||
|
||
An important design decision for Kiali was to decide that it will continue consuming data from one Prometheus instance per all clusters. For this reason, Prometheus needs to be federated, meaning that all the remote’s metrics should be fetched by the main Prometheus. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
An important design decision for Kiali was to decide that it will continue consuming data from one Prometheus instance per all clusters. For this reason, Prometheus needs to be federated, meaning that all the remote’s metrics should be fetched by the main Prometheus. | |
Even in a multi-cluster environment, Kiali expects unified metrics from a single Prometheus endpoint for all clusters. In this tutorial, we will federate the two Prometheus instances, meaning that all the remote’s metrics should be fetched by the main Prometheus. |
weight: 9 | ||
--- | ||
|
||
We will configure Kiali to access the remote cluster. This will require a secret (similar to the Istio secret) containing the credentials for Kiali to fetch information for the remote cluster: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We will configure Kiali to access the remote cluster. This will require a secret (similar to the Istio secret) containing the credentials for Kiali to fetch information for the remote cluster: | |
We will configure Kiali to access the remote cluster. This will require a secret (similar to the Istio secret) containing the credentials for Kiali to fetch information from the remote cluster: |
./kiali-prepare-remote-cluster.sh --kiali-cluster-context $CLUSTER_EAST --remote-cluster-context $CLUSTER_WEST | ||
``` | ||
|
||
Finally, upgrade the installation for Kiali to pick up the secret: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this just restart the kiali pod?
@leandroberetta it looks like my review was submitted a few minutes too late. Not sure if it will be easy to apply the suggestions now. I could open a separate PR with some of the comments/suggestions and you can take a look at it? |
Yeah, I didn't see your review at all! I think I merged before you published it. I'm ok if you want to collaborate too. Thanks. |
* tutorial * mc tutorial * fixes * addressing feedback * Update content/en/docs/Tutorials/multicluster/04-Install-Istio-east-cluster.md Co-authored-by: John Mazzitelli <[email protected]> * Apply suggestions from code review Co-authored-by: John Mazzitelli <[email protected]> --------- Co-authored-by: John Mazzitelli <[email protected]>
Multi cluster tutorial. There are things we could add, like integration with Keycloak to show permissions. Also, this is just for primary and remote, so I see in the near future adding the multi primary case or doing another one for that.
netlify link: https://deploy-preview-681--kiali.netlify.app/docs/tutorials/multicluster/
Resolves kiali/kiali#6306