Add FE Ingress

[debugger24]

What was changed

Added ingress for server frontend service

Why?

To access server frontend service from other microservice

Checklist

1. Closes [Feature Request] Add ingress for temporal server frontend #430

[CLAassistant]

All committers have signed the CLA.

[vithubati]

with this change, will you be able to connect the frontend via SDK? like below
c, err := client.Dial(client.Options{ HostPort: "ingress-path-prefix:80", })

[debugger24]

Yes and in ingress we need to mention grpc protocol grpc uses http/2

…

On Sat, 28 Oct 2023 at 20:36, Vithu R ***@***.***> wrote: with this change, will you be able to connect the frontend via SDK? like below c, err := client.Dial(client.Options{ HostPort: "ingress-path-prefix:80", }) — Reply to this email directly, view it on GitHub <#435 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADA6PH55HDZB7MV73U4IDT3YBUNPDAVCNFSM6AAAAAA6LPP3O2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBTHA2DCNJWGE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[vithubati]

I tried. but it didn't work for me. sharing my config
server:
frontend:
service:
annotations: {} # Evaluated as template
type: ClusterIP
port: 7233
ingress:
enabled: true
className: traefik
annotations:
kubernetes.io/ingress.class: traefik-external
ingress.kubernetes.io/ssl-redirect: "false"
traefik.frontend.rule.type: PathPrefix
hosts:
- "temporal.mydomain.com"
tls:
- secretName: local-mydomain-tls
hosts:
- temporal.mydomain.com

[debugger24]

Try with this, it needs some additional annotations

server:
  frontend:
    service:
      annotations:
        traefik.ingress.kubernetes.io/service.serversScheme: h2c
        traefik.ingress.kubernetes.io/service.passHostHeader: "false"
    ingress:
      enabled: true
      annotations:
        traefik.ingress.kubernetes.io/router.entrypoints: web
      hosts:
      - ""

[sandevil007]

@debugger24 I used the below config, temporal-frontend ingress gets created but it says some backend services are in unhealthy state.
temporal-web on the other hand runs just fine with ingress enabled to true. Any guidance?

 frontend:
  # replicaCount: 1
  service:
    annotations: {} # Evaluated as template
    type: ClusterIP
    port: 7233
  ingress:
    enabled: true
    annotations:
      kubernetes.io/ingress.global-static-ip-name: <static-ip>
  metrics:
    annotations:
      enabled: true
    serviceMonitor: {}
    # enabled: false
    prometheus: {}
    # timerType: histogram
  podAnnotations: {}
  podLabels: {}
  resources: {}
  nodeSelector: {}
  tolerations: []
  affinity: {}

[debugger24]

Mostly it is because this service use gRPC, depending on the ingress controller you may need to add additional annotation, like in my traefik (can be different for you) I use traefik.ingress.kubernetes.io/service.serversscheme: h2c annotation on service.

[sandevil007]

@debugger24 I'm using, gce as the ingress class, not using any ingress controller like traefik or nginx.

[ogusak]

Just found this PR and wanted to add: we implemented similar change internally and made it to work with ALB ingress over SSL for EKS on AWS. Here is the ALB ingress config we used:

    ingress:
      enabled: true
      className: "alb"
      annotations:
        alb.ingress.kubernetes.io/group.name: "<common group name to share ALB with other ingresses>"
        alb.ingress.kubernetes.io/listen-ports: "[{\"HTTPS\": 443}]"
        alb.ingress.kubernetes.io/target-type: ip
        alb.ingress.kubernetes.io/scheme: internal
        alb.ingress.kubernetes.io/load-balancer-attributes: "routing.http2.enabled=true"
        alb.ingress.kubernetes.io/target-group-attributes: "deregistration_delay.timeout_seconds=60"
        alb.ingress.kubernetes.io/certificate-arn: "<ARN of SSL cert>"
        alb.ingress.kubernetes.io/backend-protocol: HTTP
        alb.ingress.kubernetes.io/backend-protocol-version: GRPC
        alb.ingress.kubernetes.io/healthcheck-protocol: HTTP
        alb.ingress.kubernetes.io/healthcheck-path: /grpc.health.v1.Health/Check
        alb.ingress.kubernetes.io/success-codes: "0"

For the temporal client to connect successfully, we also had to set TEMPORAL_TLS_SERVER_NAME environment variable - it should be set to the domain name of the ALB and the SSL certificate installed on the ALB.

Looking forward to have this feature added to Temporal's official helm charts! Thanks!

[expertonium]

The information from @ogusak was extremely helpful. For those using Google Cloud, it can also be pointed out that the ingress facilitated by this PR is not required when electing to implement "Container-native load balancing through standalone zonal NEGs." If such an infrastructure is desired, the critical annotation is the following.

server:
  frontend:
    service:
      annotations:
        cloud.google.com/neg: '{"ingress": false, "exposed_ports": {"7233":{"name": "temporal-frontend-neg"}}}'

[robholland]

We already have an ingress: https://github.com/temporalio/helm-charts/blob/1b62283da2a94cb172f92644755348456d5ee617/charts/temporal/templates/web-ingress.yaml

[robholland]

Oops, my apologies, different service.

[robholland]

Looks good, please rename to frontend-ingress

[robholland]

Please rename file to frontend-ingress.yaml.

[debugger24]

Thanks for the catch, renamed.

[debugger24]

other resources too like service is in server-service.yaml. should we really make frontend-ingress.yaml ?

[robholland]

Yes please, I'll be adjusting file names to be more reflective as well.

[TheGeniesis]

Just my 2 cents (it took me several hours to find a working solution):
My working config for AKS cluster v1.27 with ingress-nginx/ingress-nginx chart (v4.10.0):

server:
  frontend:
    ingress:
      enabled: true    
      hosts:
        - <host>
      # TLS is taken from default secret configured for ingress chart
      annotations:
        nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
        nginx.ingress.kubernetes.io/ssl-redirect: "true"
        nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
        nginx.ingress.kubernetes.io/ssl-passthrough: "true"

Test: grpcurl <host>:443 list