gateway-rl-for-cluster-operators.md

Gateway Rate Limiting for Cluster Operators

This user guide walks you through an example of how to configure rate limiting for all routes attached to an ingress gateway.

Run the steps ① → ⑤

① Setup

This step uses tooling from the Kuadrant Operator component to create a containerized Kubernetes server locally using Kind, where it installs Istio, Kubernetes Gateway API and Kuadrant itself.

Note: In production environment, these steps are usually performed by a cluster operator with administrator privileges over the Kubernetes cluster.

Clone the project:

git clone https://github.com/Kuadrant/kuadrant-operator && cd kuadrant-operator

Setup the environment:

make local-setup

Request an instance of Kuadrant:

kubectl -n kuadrant-system apply -f - <<EOF
apiVersion: kuadrant.io/v1beta1
kind: Kuadrant
metadata:
  name: kuadrant
spec: {}
EOF

② Create the ingress gateways

kubectl -n gateway-system apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: external
  annotations:
    kuadrant.io/namespace: kuadrant-system
    networking.istio.io/service-type: ClusterIP
spec:
  gatewayClassName: istio
  listeners:
  - name: external
    port: 80
    protocol: HTTP
    hostname: '*.io'
    allowedRoutes:
      namespaces:
        from: All
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: internal
  annotations:
    kuadrant.io/namespace: kuadrant-system
    networking.istio.io/service-type: ClusterIP
spec:
  gatewayClassName: istio
  listeners:
  - name: local
    port: 80
    protocol: HTTP
    hostname: '*.local'
    allowedRoutes:
      namespaces:
        from: All
EOF

③ Enforce rate limiting on requests incoming through the external gateway

    ┌───────────┐      ┌───────────┐
    │ (Gateway) │      │ (Gateway) │
    │  external │      │  internal │
    │           │      │           │
    │   *.io    │      │  *.local  │
    └───────────┘      └───────────┘
          ▲
          │
┌─────────┴─────────┐
│ (RateLimitPolicy) │
│       gw-rlp      │
└───────────────────┘

Create a Kuadrant RateLimitPolicy to configure rate limiting:

kubectl apply -n gateway-system -f - <<EOF
apiVersion: kuadrant.io/v1
kind: RateLimitPolicy
metadata:
  name: gw-rlp
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: Gateway
    name: external
  limits:
    "global":
      rates:
      - limit: 5
        window: 10s
EOF

Note: It may take a couple of minutes for the RateLimitPolicy to be applied depending on your cluster.

④ Deploy a sample API to test rate limiting enforced at the level of the gateway

                           ┌───────────┐      ┌───────────┐
┌───────────────────┐      │ (Gateway) │      │ (Gateway) │
│ (RateLimitPolicy) │      │  external │      │  internal │
│       gw-rlp      ├─────►│           │      │           │
└───────────────────┘      │   *.io    │      │  *.local  │
                           └─────┬─────┘      └─────┬─────┘
                                 │                  │
                                 └─────────┬────────┘
                                           │
                                 ┌─────────┴────────┐
                                 │   (HTTPRoute)    │
                                 │     toystore     │
                                 │                  │
                                 │ *.toystore.io    │
                                 │ *.toystore.local │
                                 └────────┬─────────┘
                                          │
                                   ┌──────┴───────┐
                                   │   (Service)  │
                                   │   toystore   │
                                   └──────────────┘

Deploy the sample API:

kubectl apply -f examples/toystore/toystore.yaml

Route traffic to the API from both gateways:

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: toystore
spec:
  parentRefs:
  - name: external
    namespace: gateway-system
  - name: internal
    namespace: gateway-system
  hostnames:
  - "*.toystore.io"
  - "*.toystore.local"
  rules:
  - backendRefs:
    - name: toystore
      port: 80
EOF

⑤ Verify the rate limiting works by sending requests in a loop

Expose the gateways, respectively at the port numbers 9081 and 9082 of the local host:

kubectl port-forward -n gateway-system service/external-istio 9081:80 >/dev/null 2>&1 &
kubectl port-forward -n gateway-system service/internal-istio 9082:80 >/dev/null 2>&1 &

Up to 5 successful (200 OK) requests every 10 seconds through the external ingress gateway (*.io), then 429 Too Many Requests:

while :; do curl --write-out '%{http_code}\n' --silent --output /dev/null -H 'Host: api.toystore.io' http://localhost:9081 | grep -E --color "\b(429)\b|$"; sleep 1; done

Unlimited successful (200 OK) through the internal ingress gateway (*.local):

while :; do curl --write-out '%{http_code}\n' --silent --output /dev/null -H 'Host: api.toystore.local' http://localhost:9082 | grep -E --color "\b(429)\b|$"; sleep 1; done

Cleanup

make local-cleanup