From 399a072e8bf65656436f553d109b6db331303a79 Mon Sep 17 00:00:00 2001 From: John Mazzitelli Date: Tue, 3 Dec 2024 14:18:16 -0500 Subject: [PATCH] document profiler and logger (#839) * document profiler and logger fixes: https://github.com/kiali/kiali/issues/7971 * fix typo --- .../en/docs/Configuration/debugging-kiali.md | 54 ++++++++++++++++++- 1 file changed, 53 insertions(+), 1 deletion(-) diff --git a/content/en/docs/Configuration/debugging-kiali.md b/content/en/docs/Configuration/debugging-kiali.md index 673c6f0a..421b7925 100644 --- a/content/en/docs/Configuration/debugging-kiali.md +++ b/content/en/docs/Configuration/debugging-kiali.md @@ -1,8 +1,25 @@ --- title: "Debugging Kiali" -description: "How to debug Kiali using traces." +description: "How to debug Kiali using logs, traces, and profiler." --- +## Logs + +The most basic way of debugging the internals of Kiali is to examine its log messages. A typical way of examining the log messages is via: +``` +kubectl logs -n istio-system deployment/kiali +``` +Each log message is logged at a specific level. The different log levels are `trace`, `debug`, `info`, `warn`, `error`, and `fatal`. By default, log messages at `info` level and higher will be logged. If you want to see more verbose logs, set the log level to `debug` or `trace` (`trace` is the most verbose setting and will make the log output very "noisy"). You set the log level in the Kiali CR: + +```yaml +spec: + deployment: + logger: + log_level: debug +``` + +## Tracing + Kiali provides the ability to emit debugging traces to the [distributed tracing](/docs/configuration/p8s-jaeger-grafana/tracing) platform, Jaeger or Grafana Tempo. {{% alert color="warning" %}} @@ -35,4 +52,39 @@ The traces emitted by Kiali can be searched in the _Kiali_ workload: ![Kiali traces](/images/documentation/configuration/kiali_own_traces.png) +## Profiler + +The Kial Server is integrated with the Go pprof profiler. By default, the integration is disabled. If you want the Kiali Server to generate profile reports, enable it in the Kiali CR: + +```yaml +spec: + server: + profiler: + enabled: true +``` + +Once the profiler is enabled, you can access the profile reports by pointing your browser to the `/debug/pprof` endpoint and click the link to the profile report you want. You can obtain a specific profile report by appending the name of the profile to the URL. For example, if your Kiali Server is found at the root URL of "http://localhost:20001/kiali", and you want the heap profile report, the URL `http://localhost:20001/kiali/debug/pprof/heap` will provide the data for that report. + +Go provides a pprof tool that you can then use to visualize the profile report. This allows you to analyze the data to help find potential problems in the Kiali Server itself. For example, you can start the pprof UI on port 8080 which allows you to see the profile data in your browser: + +``` +go tool pprof -http :8080 http://localhost:20001/kiali/debug/pprof/heap +``` + +You can download a profile report and store it as a file for later analysis. For example: + +``` +curl -o pprof.txt http://localhost:20001/kiali/debug/pprof/heap +``` + +You can then examine the data found in the profile report: + +``` +go tool pprof -http :8080 ./pprof.txt +``` + +Your browser will be opened to `http://localhost:8080/ui` which allows you to see the profile report. + +## Kiali CR Status +When you install the Kiali Server via the Kiali Operator, you do so by creating a Kiali CR. One quick way to debug the status of a Kiali Server installation is to look at the Kiali CR's `status` field (e.g. `kubectl get kiali --all-namespaces -o jsonpath='{..status}'`). The operator will report any installation errors within this Kiali CR status. If the Kiali Server fails to install, always check the Kiali CR status field first because in many instances you will find an error message there that can provide clear guidance on what to do next.