Multicluster tutorial #681

leandroberetta · 2023-08-02T00:56:36Z

Multi cluster tutorial. There are things we could add, like integration with Keycloak to show permissions. Also, this is just for primary and remote, so I see in the near future adding the multi primary case or doing another one for that.

netlify link: https://deploy-preview-681--kiali.netlify.app/docs/tutorials/multicluster/

Resolves kiali/kiali#6306

josunect

Nice tutorial, @leandroberetta !
It is very detailed and and think it will be very useful.

I had added some comments, some are just suggestions.

josunect · 2023-08-09T07:35:46Z

content/en/docs/Tutorials/multicluster/_index.md

@@ -0,0 +1,9 @@
+---
+title: Travels Demo, Now Multicluster
+description: Learn how to configure and use Kiali in a Istio multicluster scenario.


in an Istio

Maybe we can link here? https://deploy-preview-681--kiali.netlify.app/docs/configuration/multi-cluster/ I think is useful to have related documentation linked, in case the reader needs more contexts about multi cluster.

Also, sometimes is confusing for me, how to write the Multi cluster word :) But as in the related link is written as multi-cluster, it would make sense to be consistent also here?

Mentioning the docs is something important, yes, I will add that.

Regarding to the multicluster word, I saw that in Istio, they called "multicluster", but I can change it, for the sake of consistency.

I think "multicluster" is fine - especially if that's how Istio uses it. It definitely isn't two words (not "multi cluster") but I have seen it with a hypen ("multi-cluster"). I could go either way - either one word or with the hyphen (not two words though - "multi" isn't really a word in English).

josunect · 2023-08-09T07:57:17Z

content/en/docs/Tutorials/multicluster/04-Install-Istio-east-cluster.md

+istioctl install -y --set values.pilot.env.EXTERNAL_ISTIOD=true --context=$CLUSTER_EAST -f $CLUSTER_EAST.yaml
+```
+
+After the installation, we need to create what we called an “east-west” gateway. It’s an ingress gateway just for the cross cluster configuration as we are opting to use the installation for different networks (this will be the case in the majority of the productive scenarios). 


production scenarios?

Yeah probably :)

josunect · 2023-08-09T08:03:40Z

content/en/docs/Tutorials/multicluster/05-Install-Kiali.md

+kubectl config use-context $CLUSTER_EAST
+
+helm upgrade --install --namespace istio-system --set kubernetes_config.cache_enabled=false --set auth.strategy=anonymous --set deployment.logger.log_level=debug --set deployment.ingress.enabled=true --repo https://kiali.org/helm-charts kiali-server kiali-server 
+```


Just a suggestion, maybe we can also say how to verify that Kiali is running?
Something like accessing Kiali in http://..

josunect · 2023-08-09T08:15:40Z

content/en/docs/Tutorials/multicluster/08-Install-Istio-west-cluster.md

+weight: 8
+---
+
+This installation will be different as this cluster will be a remote:


Maybe it is possible to explain a bit why is different? Like the control plane will not be installed, just the CRDs, and it will be managed by a external CP?

josunect · 2023-08-09T08:16:48Z

content/en/docs/Tutorials/multicluster/08-Install-Istio-west-cluster.md

+Finally, we create the east-west gateway
+
+```
+samples/multicluster/gen-eastwest-gateway.sh \


Shouldn't be?
$ISTIO_DIR/samples/...

josunect · 2023-08-09T09:00:21Z

content/en/docs/Tutorials/multicluster/09-Configure-Kiali.md

+Finally, upgrade the installation for Kiali to pick up the secret:
+
+```
+helm upgrade \ 


I had some issues in this line:

Error: "helm upgrade" requires 2 arguments

Probably an indentation error, but if I put this in the same line:

Error: UPGRADE FAILED: "kiali-server" has no deployed releases

I guess that should be in istio-system namespace, in this case I get:

Error: UPGRADE FAILED: cannot patch "kiali" with kind ClusterRoleBinding: ClusterRoleBinding.rbac.authorization.k8s.io "kiali" is invalid: roleRef: Invalid value: rbac.RoleRef{APIGroup:"rbac.authorization.k8s.io", Kind:"ClusterRole", Name:"kiali-viewer"}: cannot change roleRef

I will check this, it's rare because I tested this but I will double check.

jmazzitelli

overall, very nice. worked all the way through (well, except I needed to pass in --driver kvm2 when starting minikube to avoid getting some startup warnings, but that could be an issue with my own setup)

content/en/docs/Tutorials/multicluster/04-Install-Istio-east-cluster.md

content/en/docs/Tutorials/multicluster/05-Install-Kiali.md

content/en/docs/Tutorials/multicluster/06-Install-Travels-east-cluster.md

content/en/docs/Tutorials/multicluster/08-Install-Istio-west-cluster.md

content/en/docs/Tutorials/multicluster/09-Configure-Kiali.md

content/en/docs/Tutorials/multicluster/10-Install-Travels-west-cluster copy.md

…luster.md Co-authored-by: John Mazzitelli <[email protected]>

Co-authored-by: John Mazzitelli <[email protected]>

jmazzitelli

lgtm!

nrfox

The tutorial is great, thanks for putting all this together. Just a few suggestions/questions.

nrfox · 2023-08-10T11:32:52Z

content/en/docs/Tutorials/multicluster/_index.md

@@ -0,0 +1,10 @@
+---
+title: Travels Demo, Now Multicluster


Maybe remove the Now part and just have this be:

Suggested change

title: Travels Demo, Now Multicluster

title: Travels Demo Multicluster

Users may not do the tutorial sequentially or they might skip the single cluster tutorial altogether.

nrfox · 2023-08-10T12:21:49Z

content/en/docs/Tutorials/multicluster/01-Introduction.md

I don't think you need to go into so much detail on the primary-remote deployment model. I think it's enough to link to the istio docs and then provide more detail here on the setup in this tutorial i.e. primary-remote, multiple networks, federated prometheus, anonymous auth.

Maybe it would also be helpful to have somewhere a "jump to installing Kiali" section if you are trying to run the tutorial on an existing primary-remote deployment and don't want to go through all the minikube setup.

nrfox · 2023-08-10T12:22:50Z

content/en/docs/Tutorials/multicluster/02-Prerequisites.md

+* Istio v1.18.1
+* Kiali v1.70
+
+Clusters are provided by minikube instances, but we can choose others instead, like OpenShift or just vanilla Kubernetes installations.


Suggested change

Clusters are provided by minikube instances, but we can choose others instead, like OpenShift or just vanilla Kubernetes installations.

Clusters are provided by minikube instances, but this tutorial should work on on any Kubernetes environment.

nrfox · 2023-08-10T12:29:14Z

content/en/docs/Tutorials/multicluster/02-Prerequisites.md

+CLUSTER_EAST="east"
+CLUSTER_WEST="west"
+ISTIO_DIR="absolute-path-to-istio-folder"


Assuming a bash shell, do these need export in front of them in order to be set for the following commands?

nrfox · 2023-08-10T12:38:17Z

content/en/docs/Tutorials/multicluster/04-Install-Istio-east-cluster.md

+      network: network1
+EOF
+
+istioctl install -y --set values.pilot.env.EXTERNAL_ISTIOD=true --context=$CLUSTER_EAST -f $CLUSTER_EAST.yaml


This fails if you don't have istioctl in your PATH. I think $ISTIO_DIR/bin/istioctl would work though? It's also fine if we expect istioctl to be in your PATH.

nrfox · 2023-08-10T12:44:51Z

content/en/docs/Tutorials/multicluster/06-Install-Travels-east-cluster.md

+
+So far, we installed everything on one cluster, similarly to the Travels tutorial for a single cluster.
+
+Now we will expand this topology to include a remote cluster. As we commented this situation can be very common in a production scenario, either because we might want to split some applications into different clusters, generally because they are maintained by different developers or for high availability or just making applications available in other zones to reduce latencies.


As with the intro, I'm not sure we need to include motivations for adopting primary-remote here in the tutorial.

I think it's ok to give people some sort of use cases, but I'm ok if you prefer to remove it. I'm working on a blog post that might include thise motivations.

nrfox · 2023-08-10T12:48:02Z

content/en/docs/Tutorials/multicluster/07-Deploy-west-cluster.md

+Run the following commands to deploy the second cluster:
+
+```
+minikube start -p $CLUSTER_WEST --network istio --memory 8g --cpus 4


I get this error when running with the docker driver (the default when you don't provide one):

Startup with docker driver failed, trying with alternate driver kvm2: Failed to start host: can't create with that IP, address already in use

it then switches to kvm2 because I have that installed on my system but if I didn't the tutorial might fail at that point altogether.

nrfox · 2023-08-10T12:54:44Z

content/en/docs/Tutorials/multicluster/08-Install-Istio-west-cluster.md

+
+## Prometheus federation
+
+An important design decision for Kiali was to decide that it will continue consuming data from one Prometheus instance per all clusters. For this reason, Prometheus needs to be federated, meaning that all the remote’s metrics should be fetched by the main Prometheus.


Suggested change

An important design decision for Kiali was to decide that it will continue consuming data from one Prometheus instance per all clusters. For this reason, Prometheus needs to be federated, meaning that all the remote’s metrics should be fetched by the main Prometheus.

Even in a multi-cluster environment, Kiali expects unified metrics from a single Prometheus endpoint for all clusters. In this tutorial, we will federate the two Prometheus instances, meaning that all the remote’s metrics should be fetched by the main Prometheus.

nrfox · 2023-08-10T12:56:11Z

content/en/docs/Tutorials/multicluster/09-Configure-Kiali.md

+weight: 9
+---
+
+We will configure Kiali to access the remote cluster. This will require a secret (similar to the Istio secret) containing the credentials for Kiali to fetch information for the remote cluster:


Suggested change

We will configure Kiali to access the remote cluster. This will require a secret (similar to the Istio secret) containing the credentials for Kiali to fetch information for the remote cluster:

We will configure Kiali to access the remote cluster. This will require a secret (similar to the Istio secret) containing the credentials for Kiali to fetch information from the remote cluster:

nrfox · 2023-08-10T12:57:03Z

content/en/docs/Tutorials/multicluster/09-Configure-Kiali.md

+./kiali-prepare-remote-cluster.sh --kiali-cluster-context $CLUSTER_EAST --remote-cluster-context $CLUSTER_WEST
+```
+
+Finally, upgrade the installation for Kiali to pick up the secret:


Does this just restart the kiali pod?

nrfox · 2023-08-10T13:36:53Z

@leandroberetta it looks like my review was submitted a few minutes too late. Not sure if it will be easy to apply the suggestions now. I could open a separate PR with some of the comments/suggestions and you can take a look at it?

leandroberetta · 2023-08-10T18:14:39Z

@leandroberetta it looks like my review was submitted a few minutes too late. Not sure if it will be easy to apply the suggestions now. I could open a separate PR with some of the comments/suggestions and you can take a look at it?

Yeah, I didn't see your review at all! I think I merged before you published it. I'm ok if you want to collaborate too. Thanks.

* tutorial * mc tutorial * fixes * addressing feedback * Update content/en/docs/Tutorials/multicluster/04-Install-Istio-east-cluster.md Co-authored-by: John Mazzitelli <[email protected]> * Apply suggestions from code review Co-authored-by: John Mazzitelli <[email protected]> --------- Co-authored-by: John Mazzitelli <[email protected]>

leandroberetta added 3 commits August 1, 2023 21:54

tutorial

bd2eca7

mc tutorial

fe911df

fixes

3ca91a7

leandroberetta requested review from josunect and nrfox August 8, 2023 17:39

josunect reviewed Aug 9, 2023

View reviewed changes

addressing feedback

0092488

leandroberetta marked this pull request as ready for review August 9, 2023 12:53

jmazzitelli requested changes Aug 9, 2023

View reviewed changes

leandroberetta and others added 2 commits August 9, 2023 16:08

Update content/en/docs/Tutorials/multicluster/04-Install-Istio-east-c…

5799c69

…luster.md Co-authored-by: John Mazzitelli <[email protected]>

Apply suggestions from code review

fb4254e

Co-authored-by: John Mazzitelli <[email protected]>

jmazzitelli approved these changes Aug 9, 2023

View reviewed changes

leandroberetta self-assigned this Aug 10, 2023

leandroberetta added the enhancement New feature or request label Aug 10, 2023

leandroberetta merged commit c83d3cd into kiali:staging Aug 10, 2023
4 of 5 checks passed

nrfox requested changes Aug 10, 2023

View reviewed changes

nrfox mentioned this pull request Aug 22, 2023

Mc tutorial suggestions #689

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Multicluster tutorial #681

Multicluster tutorial #681

leandroberetta commented Aug 2, 2023 •

edited

Loading

josunect left a comment

josunect Aug 9, 2023

josunect Aug 9, 2023

leandroberetta Aug 9, 2023

jmazzitelli Aug 9, 2023

josunect Aug 9, 2023

leandroberetta Aug 9, 2023

josunect Aug 9, 2023

josunect Aug 9, 2023

josunect Aug 9, 2023

josunect Aug 9, 2023

leandroberetta Aug 9, 2023

jmazzitelli left a comment

jmazzitelli left a comment

nrfox left a comment

nrfox Aug 10, 2023

nrfox Aug 10, 2023

nrfox Aug 10, 2023

nrfox Aug 10, 2023

nrfox Aug 10, 2023

nrfox Aug 10, 2023

nrfox Aug 10, 2023

leandroberetta Aug 10, 2023

nrfox Aug 10, 2023

nrfox Aug 10, 2023

nrfox Aug 10, 2023

nrfox Aug 10, 2023

nrfox commented Aug 10, 2023

leandroberetta commented Aug 10, 2023

	title: Travels Demo, Now Multicluster
	title: Travels Demo Multicluster

	Clusters are provided by minikube instances, but we can choose others instead, like OpenShift or just vanilla Kubernetes installations.
	Clusters are provided by minikube instances, but this tutorial should work on on any Kubernetes environment.


		So far, we installed everything on one cluster, similarly to the Travels tutorial for a single cluster.

		Now we will expand this topology to include a remote cluster. As we commented this situation can be very common in a production scenario, either because we might want to split some applications into different clusters, generally because they are maintained by different developers or for high availability or just making applications available in other zones to reduce latencies.


		## Prometheus federation

		An important design decision for Kiali was to decide that it will continue consuming data from one Prometheus instance per all clusters. For this reason, Prometheus needs to be federated, meaning that all the remote’s metrics should be fetched by the main Prometheus.

	An important design decision for Kiali was to decide that it will continue consuming data from one Prometheus instance per all clusters. For this reason, Prometheus needs to be federated, meaning that all the remote’s metrics should be fetched by the main Prometheus.
	Even in a multi-cluster environment, Kiali expects unified metrics from a single Prometheus endpoint for all clusters. In this tutorial, we will federate the two Prometheus instances, meaning that all the remote’s metrics should be fetched by the main Prometheus.

	We will configure Kiali to access the remote cluster. This will require a secret (similar to the Istio secret) containing the credentials for Kiali to fetch information for the remote cluster:
	We will configure Kiali to access the remote cluster. This will require a secret (similar to the Istio secret) containing the credentials for Kiali to fetch information from the remote cluster:

Multicluster tutorial #681

Multicluster tutorial #681

Conversation

leandroberetta commented Aug 2, 2023 • edited Loading

josunect left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jmazzitelli left a comment

Choose a reason for hiding this comment

jmazzitelli left a comment

Choose a reason for hiding this comment

nrfox left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nrfox commented Aug 10, 2023

leandroberetta commented Aug 10, 2023

leandroberetta commented Aug 2, 2023 •

edited

Loading