Configurable grafana endpoint in web app

[markwinter]

/kind feature

The web app currently seems to assume that grafana will be accessible via the same ingress as kubeflow under the path /grafana. We have a separated prometheus+grafana that we would like to use instead.

In the web app, I can see that grafanaPrefix is pulled out into the environment.ts file, but I think these vars are only read/replaced at build time and can't be overwritten at run time - I don't know Angular

I think the var needs to be read from the kfserving-models-web-app-config configMap instead? Like APP_PREFIX etc.

@kimwnasptd could you take a look please?

[markwinter]

For now I have just created a VirtualService but I think a configurable grafana endpoint seems more correct?

[markwinter]

It would also be helpful if the DBs were configurable too

db/knative-serving-revision-cpu-and-memory-usage
db/knative-serving-revision-http-requests

[yuzisun]

/assign @kimwnasptd

[kimwnasptd]

@markwinter while the mechanism for allowing admins to configure different aspects of the web app, via a mounted ConfigMap, is something we want to implement I'd like to understand a little bit more your specific use case.

For now I have just created a VirtualService but I think a configurable grafana endpoint seems more correct?

You will always need to create a VirtualService to expose the Grafana instance outside the cluster. In this vsvc you can also set the prefix accordingly.

What is the limitation you are bumping into by having the web app always expect Grafana to be exposed under a /grafana prefix?