Skip to content

Commit ec5ddee

Browse files
oyyblinoyyblin
committed
feat: ignore upstream cors headers; add websocket support
1 parent f709e64 commit ec5ddee

File tree

3 files changed

+97
-41
lines changed

3 files changed

+97
-41
lines changed

‎README.md

+25-18
Original file line numberDiff line numberDiff line change
@@ -17,17 +17,17 @@ The solution uses a two-tier architecture:
1717

1818
1. **Nginx (Front Tier)**
1919

20-
- Handles incoming HTTP/HTTPS traffic
21-
- Provides initial request routing
22-
- Manages SSL/TLS termination
23-
- Forwards requests to HAProxy
20+
- Handles incoming traffic on port 8080
21+
- Automatic WebSocket protocol detection and routing
22+
- Manages CORS and HTTP headers
23+
- Routes traffic to appropriate HAProxy ports (8545/8546)
2424

2525
2. **HAProxy (Back Tier)**
26-
- Manages load balancing across RPC nodes
27-
- Implements sticky sessions
28-
- Performs health checks
29-
- Handles WebSocket connections
30-
- Provides failover capabilities
26+
- Listens on ports 8545 (RPC) and 8546 (WebSocket)
27+
- Port-based backend selection
28+
- Maintains sticky sessions for 30 minutes
29+
- Load balances across 4 backend nodes
30+
- Performs TCP-level health checks every 3 seconds
3131

3232
## Prerequisites
3333

@@ -61,20 +61,27 @@ The solution uses a two-tier architecture:
6161

6262
The HAProxy configuration (`haproxy.cfg`) includes:
6363

64-
- TCP mode for WebSocket support
65-
- Round-robin load balancing
66-
- Sticky sessions based on client IP
67-
- Health checks for RPC nodes
68-
- Configurable timeouts and connection limits
64+
- Dual-port TCP mode (8545/8546) for RPC and WebSocket support
65+
- Port-based traffic routing between RPC and WebSocket backends
66+
- Round-robin load balancing with sticky sessions
67+
- Sticky sessions based on client IP (30-minute persistence)
68+
- Health checks every 3 seconds with 2-attempt failure/rise thresholds
69+
- Configurable timeouts (connect: 5s, client: 50s, server: 50s)
70+
- Internal health monitoring endpoint on localhost:8547
6971

7072
### Nginx Configuration
7173

7274
The Nginx configuration (`nginx.conf`) provides:
7375

74-
- Reverse proxy setup
75-
- Header forwarding
76-
- Health check endpoint
77-
- Connection pooling
76+
- Single entry point on port 8080
77+
- Automatic WebSocket protocol upgrade handling
78+
- CORS headers management with preflight request support
79+
- Smart traffic routing:
80+
- WebSocket connections → HAProxy port 8546
81+
- RPC traffic → HAProxy port 8545
82+
- Dual health check endpoints:
83+
- `/health/alive` for basic liveness checks
84+
- `/health/ready` for HAProxy status verification
7885

7986
## Health Checks
8087

‎haproxy.cfg

+29-6
Original file line numberDiff line numberDiff line change
@@ -13,15 +13,20 @@ defaults
1313
timeout client 50s
1414
timeout server 50s
1515

16-
# Main RPC frontend - handles incoming blockchain RPC requests
17-
frontend rpc_frontend
18-
# Listen on all interfaces for external access
16+
# Combined frontend for both HTTP/RPC and WebSocket traffic
17+
frontend combined_frontend
1918
bind 0.0.0.0:8545
20-
# TCP mode for WebSocket support
19+
bind 0.0.0.0:8546
2120
mode tcp
21+
22+
# Use a TCP mode ACL to detect WebSocket traffic (port 8546)
23+
acl is_websocket dst_port 8546
24+
25+
# Route based on the destination port
26+
use_backend ws_backend if is_websocket
2227
default_backend rpc_backend
2328

24-
# Backend configuration for RPC nodes
29+
# Backend configuration for RPC nodes (port 8545)
2530
backend rpc_backend
2631
mode tcp
2732
# Round robin load balancing between nodes
@@ -39,11 +44,29 @@ backend rpc_backend
3944
server node3 54.212.168.35:8545 check inter 3s fall 2 rise 2
4045
server node4 54.202.224.179:8545 check inter 3s fall 2 rise 2
4146

47+
# Backend configuration for WebSocket nodes (port 8546)
48+
backend ws_backend
49+
mode tcp
50+
# Round robin load balancing between nodes
51+
balance roundrobin
52+
# Enable TCP health checks
53+
option tcp-check
54+
# Allow server retry on failure
55+
option redispatch
56+
# Sticky sessions configuration
57+
stick-table type ip size 200k expire 30m
58+
stick on src
59+
# WebSocket nodes with health checks
60+
server node1 35.89.200.123:8546 check inter 3s fall 2 rise 2
61+
server node2 18.236.243.137:8546 check inter 3s fall 2 rise 2
62+
server node3 54.212.168.35:8546 check inter 3s fall 2 rise 2
63+
server node4 54.202.224.179:8546 check inter 3s fall 2 rise 2
64+
4265
# Health check endpoint for monitoring HAProxy status
4366
# Only accessible from localhost for security
4467
frontend health_check
4568
# Bind only to localhost (127.0.0.1) to prevent external access
46-
bind 127.0.0.1:8546
69+
bind 127.0.0.1:8547
4770
# Use HTTP mode since health checks are HTTP requests
4871
mode http
4972
# Create a simple health endpoint at /health

‎nginx.conf

+43-17
Original file line numberDiff line numberDiff line change
@@ -4,33 +4,59 @@ events {
44
}
55

66
http {
7+
map $http_upgrade $connection_upgrade {
8+
default upgrade;
9+
'' close;
10+
}
11+
712
# Upstream definition for HAProxy RPC load balancer
813
upstream rpc_nodes {
914
server 127.0.0.1:8545;
1015
}
1116

17+
upstream ws_nodes {
18+
server 127.0.0.1:8546;
19+
}
20+
1221
server {
1322
listen 8080;
1423

1524
# Main location for RPC traffic
1625
location / {
17-
# Enable CORS from any origin
18-
# add_header 'Access-Control-Allow-Origin' '*' always;
19-
# add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS' always;
20-
# add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Authorization' always;
26+
# Remove any existing CORS headers from upstream
27+
proxy_hide_header 'Access-Control-Allow-Origin';
28+
proxy_hide_header 'Access-Control-Allow-Methods';
29+
proxy_hide_header 'Access-Control-Allow-Headers';
30+
proxy_hide_header 'Access-Control-Max-Age';
2131

22-
# # Handle preflight requests
23-
# if ($request_method = 'OPTIONS') {
24-
# add_header 'Access-Control-Allow-Origin' '*';
25-
# add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
26-
# add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Authorization';
27-
# add_header 'Access-Control-Max-Age' 1728000;
28-
# add_header 'Content-Type' 'text/plain; charset=utf-8';
29-
# add_header 'Content-Length' 0;
30-
# return 204;
31-
# }
32-
33-
# Forward requests to HAProxy
32+
# Handle preflight requests
33+
if ($request_method = 'OPTIONS') {
34+
add_header 'Access-Control-Allow-Origin' '*';
35+
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
36+
add_header 'Access-Control-Allow-Headers' 'Content-Type';
37+
add_header 'Access-Control-Max-Age' 1728000;
38+
add_header 'Content-Type' 'text/plain; charset=utf-8';
39+
add_header 'Content-Length' 0;
40+
return 204;
41+
}
42+
43+
# Add CORS headers for non-OPTIONS requests
44+
if ($request_method != 'OPTIONS') {
45+
add_header 'Access-Control-Allow-Origin' '*';
46+
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
47+
add_header 'Access-Control-Allow-Headers' 'Content-Type';
48+
}
49+
50+
# WebSocket specific configuration
51+
proxy_http_version 1.1;
52+
proxy_set_header Upgrade $http_upgrade;
53+
proxy_set_header Connection $connection_upgrade;
54+
55+
# Route WebSocket traffic to ws_nodes, regular traffic to rpc_nodes
56+
if ($http_upgrade = "websocket") {
57+
proxy_pass http://ws_nodes;
58+
}
59+
3460
proxy_pass http://rpc_nodes;
3561
proxy_set_header Host $host;
3662
proxy_set_header X-Real-IP $remote_addr;
@@ -51,7 +77,7 @@ http {
5177
# Short timeouts for quick health check responses
5278
proxy_connect_timeout 2s;
5379
proxy_read_timeout 2s;
54-
proxy_pass http://127.0.0.1:8546/health;
80+
proxy_pass http://127.0.0.1:8547/health;
5581
add_header Content-Type text/plain;
5682
}
5783
}

0 commit comments

Comments
 (0)