Automatic connection to Prometheus doesn't work on k3s from outside the cluster

[aantn]

           INFO     Trying to connect to Prometheus for k3d-test-krr cluster                                                                                prometheus_metrics_service.py:67
           DEBUG    Trying to find service with label selector app=kube-prometheus-stack-prometheus                                                                  service_discovery.py:73
           DEBUG    Found service with label selector app=kube-prometheus-stack-prometheus                                                                           service_discovery.py:76
           INFO     Using Prometheus at https://0.0.0.0:43893/api/v1/namespaces/default/services/robusta-kube-prometheus-st-prometheus:9090/proxy for       prometheus_metrics_service.py:96
                    cluster k3d-test-krr                                                                                                                                                    
           INFO     Prometheus not found: Couldn't connect to Prometheus found under                                                                                            loader.py:67
                    https://0.0.0.0:43893/api/v1/namespaces/default/services/robusta-kube-prometheus-st-prometheus:9090/proxy                                                               
                    Caused by HTTPError: 401 Client Error: Unauthorized for url:                                                                                                            
                    https://0.0.0.0:43893/api/v1/namespaces/default/services/robusta-kube-prometheus-st-prometheus:9090/proxy/api/v1/query)                                                 
           ERROR    Wasn't able to connect to any Prometheus service in k3d-test-krr cluster                                                                                    runner.py:65
                    Try using port-forwarding and/or setting the url manually (using the -p flag.).                                                                                         
                    For more information, see 'Giving the Explicit Prometheus URL' at https://github.com/robusta-dev/krr?tab=readme-ov-file#usage

Connecting to the same Prometheus works:

1. When krr is run locally and connects over port-forward
2. When krr is run in-cluster for k3s

I investigated this with @pavangudiwada and the issue is that there are no auth headers added for the APIServer proxy. (For other clusters like GKE we add a {'authorization': 'Bearer... } which doesn't get added for k3s for some reason.