Skip to content

Latest commit

 

History

History
63 lines (45 loc) · 4.55 KB

gue-header.md

File metadata and controls

63 lines (45 loc) · 4.55 KB
Raw

GUE Header usage in GLB

GLB Director uses Generic UDP Encapsulation to encapsulate received IP packets and send them over the wire to proxy servers. GUE provides an extensible header with an unspecified private data area, and GLB Director uses this area to encode the additional servers which may be given a second chance at utilising the given packet in the case that the current server believes it is invalid (not for a given local connection)

 0                   1                   2                   3  
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\
|          Source port          |        Destination port       | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ UDP
|             Length            |            Checksum           | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+/
| 0 |C|   Hlen  |  Proto/ctype  |             Flags             | GUE
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|     Private data type (0)     |  Next hop idx |   Hop count   |\
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
|                             Hop 0                             | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ GLB
|                              ...                              | private
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ data
|                             Hop N                             | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+/

UDP fields

GLB Director currently uses a UDP Destination port of 19523 (historically a pre-GUE version used internally at GitHub used 19522 which is LB's bytes decoded as a uint16, so the GUE version incremented this number).

Source port is used to encourage flows to be steered to different ECMP paths and different NIC RX queues. GLB Director hashes part of the header to use as the Source port. The hash must be consistent for an entire TCP flow to avoid packet reordering.

GUE fields

The GUE version, C-bit and Flags fields are all set to 0. The length of the private data is calculated from the standard GUE header length as defined in the GUE draft RFC.

The Proto/ctype field either encodes the value 4 for IPv4 or 41 for IPv6 based on the original packet received by the GLB Director, no other protocol types are used. These are the standard IANA Assigned Internet Protocol Numbers for IPv4 and IPv6 encapsulation.

The built in Linux Foo-over-UDP module supports GUE, and is compatible with this header format by ignoring the private data entirely. The GLB-REDIRECT iptables module understands how to parse and forward on this header - in the case where a packet is deemed usable by the local server it is allowed through to the Linux network stack for further processing by the gue tunnel.

Private data fields

The Private data type is 0, this is essentially a reserved field that leaves room for additional fields in future versions of the GLB header format.

The Next hop idx gives the index to the next hop to try if the local machine is unable to process the packet due to it being invalid for all established connections (and not a SYN packet). GLB Director sets this field to 0.

The Hop count specifies the number of IPv4 addresses that follow.

Packet Processing

Each server receiving a GLB GUE packet must take the following steps:

  1. If the packet is a SYN packet or relates to an existing, established or in progress connection, handle it locally by decapsulating the packet.
  2. If the packet is otherwise invalid locally, forward the packet to the next alternate server:
    1. If Next hop idx >= Hop count, we have no further alternate servers to try, drop the packet (or handle it locally).
    2. Take the outer IP packet Destination IP address and store it in Source IP address field.
    3. Take the hop specified by Next hop idx and store it in the Destination IP address field.
    4. Increment the Next hop idx field.
    5. Re-route the packet through the kernel IP stack to forward the packet to the destination.