NAT Hole Punching and Sliding-Window UDP for Real-Time WAN Tensor Transport

In distributed computing, it is easy to assume an ideal network: flat private IP addressing, static port mappings, zero packet loss, and low latency. In that fantasy world, two machines communicate simply by binding a socket to an IP and calling connect().

The real-world consumer internet is nothing like that.

Every residential broadband connection, cellular 5G mobile hotspot, and corporate office sits behind multiple layers of Network Address Translation (NAT), Carrier-Grade NAT (CGNAT), and stateful firewalls. Devices do not have public IP addresses; they cannot receive inbound connections without prior outbound traffic; and home users cannot realistically be expected to navigate their router’s admin portal to configure port forwarding or set up overlay VPNs like Tailscale or WireGuard.

When building SwarmInfer—a decentralized swarm inference engine designed to distribute 70B+ LLM layers across consumer machines—we needed two arbitrary machines anywhere in the world to connect directly and stream multi-megabyte activation tensors with sub-second latency.

Here is how we solved NAT hole punching and engineered a pipelined sliding-window UDP transport tailored for distributed neural network inference.

1. Why TCP Fails for Real-Time WAN Inference

The instinctive choice for network programming is TCP. TCP provides reliable, in-order byte streaming with congestion control built right into the OS kernel.

However, TCP possesses fundamental architectural flaws when applied to distributed tensor streaming over lossy consumer WANs:

1. Head-of-Line Blocking: If a single packet in a 10MB tensor payload is dropped by a Wi-Fi hop or 5G tower, TCP halts delivery of all subsequent packets until the missing packet is retransmitted. The receiving worker cannot begin decompressing or parsing any part of the tensor, stalling the entire pipeline.
2. Connection Fragility on Cellular IP Roaming: When a laptop switches from home Wi-Fi to a 5G hotspot, its IP address changes. A TCP socket immediately breaks with ECONNRESET or hangs indefinitely, crashing the cluster.
3. NAT Hole Punching Difficulty: Establishing a direct peer-to-peer TCP connection through symmetric NATs without a relay proxy is notoriously unreliable due to complex TCP three-way handshake state tracking in consumer router firewalls.

For SwarmInfer, we discarded TCP in favor of UDP with a custom application-level Sliding-Window ARQ protocol.

2. The NAT Problem: Cones vs. Symmetric Firewalls

When an internal device with IP 192.168.1.50 sends a packet from port 5000 to an external server, the NAT router rewrites the source address to the router’s public IP (203.0.113.10) and assigns an external port:

Internal Device [192.168.1.50:5000]
         │
         ▼ (Outbound Packet)
NAT Router [Maps to Public 203.0.113.10 : Port ???]
         │
         ▼
Internet Destination

NAT behaviors vary significantly depending on how the router assigns external ports:

• Cone NATs (Full Cone, Restricted Cone, Port-Restricted): The router assigns the same external port (e.g. 45000) for all outbound traffic originating from 192.168.1.50:5000, regardless of the destination.
• Symmetric NATs (Common in 5G and Enterprise Networks): The router assigns a different, unpredictable external port for every unique destination IP/port combination. If you query a STUN server from port 5000, the router maps it to 45000. But when you send a packet to another peer from that same local port, the router maps it to 45001, 45080, or a randomized port.

Standard hole-punching fails on symmetric NATs because Peer A cannot predict which external port Peer B’s router assigned.

3. Decentralized Peer Rendezvous via BitTorrent Trackers

Before two peers can punch holes, they must discover each other without relying on a expensive, centralized proprietary signaling server.

We utilized BitTorrent BEP 15 UDP Trackers:

1. When a user runs swarminfer serve --model llama-3-70b, the coordinator hashes the model configuration into a 20-byte info_hash (similar to a BitTorrent magnet link).
2. The node sends a lightweight UDP announce packet to public tracker pools (e.g., OpenBitTorrent, Opentracker).
3. Joining workers running swarminfer join query the same tracker hash. The tracker responds with a list of active public (IP, port) pairs of other swarm members.

This delivers instantaneous, decentralized peer discovery with zero hosting infrastructure.

4. Bypassing Symmetric Firewalls: Port-Window Sweeping

To establish a direct connection through symmetric and CGNAT routers, SwarmInfer combines RFC 5389 STUN with RFC 5128 Port-Window Sweeping:

[Peer A]                                              [Peer B]
   │                                                     │
   ├── Query STUN Server ───────────────────────────────▶│ (Discovers Public IP & Port P_A)
   │                                                     │
   │◀── Exchange Candidate Ports via Tracker ───────────▶│
   │                                                     │
   │==== Simultaneous UDP Probing (Port Window Sweeping) ====│
   ├─ Send UDP probe to (IP_B, P_B - 64)                ─┤
   ├─ Send UDP probe to (IP_B, P_B - 63) ...            ─┤
   ├─ Send UDP probe to (IP_B, P_B + 64)                ─┤
   │                                                     │
   │               <<< HOLE PUNCHED >>>                  │
   │◀═════════════ Direct Bidirectional UDP ════════════▶│

1. STUN Resolution: Both peers query a public STUN server to discover their public IP and the baseline port assigned by their respective NAT routers ($P_A$ and $P_B$).
2. Delta Analysis: By querying multiple STUN servers sequentially, the client determines whether its NAT allocates ports sequentially ($\Delta = +1$ or $+2$) or randomly.
3. Port-Window Sweeping ($\pm 64$ Ports): Both peers simultaneously send lightweight UDP probe bursts across a window of adjacent ports around the baseline port ($P \pm 64$).
4. Outbound State Creation: Even though incoming probes are initially dropped by the firewall, the outbound probes create state table entries in each router. As soon as one probe matches an outbound state, the firewall allows the packet through, completing the bidirectional hole punch.

This technique successfully traverses over 88% of symmetric consumer NATs without port forwarding and without relay proxies.

5. Pipelined Sliding-Window UDP ($W=64$)

Once direct UDP transport is open, the system must stream multi-megabyte activation tensors reliably. Raw UDP provides no delivery or ordering guarantees.

We engineered a custom Sliding-Window ARQ protocol:

17-Byte Binary Framing

Every UDP packet is prepended with an efficient 17-byte binary header:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|   Magic Byte  |   Packet Type |           Sequence ID         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                           Offset                              |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                        Payload Length                         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                           CRC32 Checksum                      |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                        Tensor Payload ...                     |

• Magic Byte (0x53): Fast sanity check to reject malformed packets immediately.
• Packet Type: Flags for DATA, ACK, NACK, HEARTBEAT, and SKIP.
• Sequence ID (16-bit): Identifies the specific forward-pass token step.
• Offset (32-bit): Byte offset of this chunk within the larger tensor.
• CRC32 Checksum (32-bit): Hardware-accelerated cyclic redundancy check to detect packet corruption caused by noisy Wi-Fi/cellular hops.

Pipelined Sliding-Window ($W=64$)

Rather than waiting for an acknowledgment after every packet (Stop-and-Wait), the sender blasts up to 64 in-flight packets before pausing:

• Packets are buffered in a ring queue.
• The receiver sends cumulative ACK frames alongside selective NACK ranges for dropped packets.
• Upon receiving a NACK, the sender retransmits only the missing packet chunks without stalling the rest of the transmission window.

6. Real-World WAN Benchmarks

We benchmarked this custom sliding-window UDP transport against standard TCP and WebSocket transports across an asymmetric WAN setup:

• Node A: M3 Max MacBook Pro running on residential Wi-Fi (Home broadband).
• Node B: Remote workstation with RTX 4080 connected via a 5G cellular phone hotspot.

By eliminating TCP head-of-line blocking and routing directly peer-to-peer, SwarmInfer streams activation tensors across the open internet in under 300ms, making real-time interactive token generation feasible over consumer connections.

7. Key Takeaways for Systems Engineers

1. Don’t Rely on Data Center Assumptions: The open internet requires resilient, peer-to-peer primitives capable of handling asymmetric bandwidth, variable latency, and dynamic IP changes.
2. Hole Punching is Achievable: RFC 5389 STUN combined with port-window sweeping reliably bypasses most consumer firewalls without requiring users to configure complex VPNs.
3. Application-Specific Transports Beat Generic TCP: For large, continuous binary payloads like neural network activations, a lightweight, selective-retransmission UDP sliding-window protocol drastically outperforms TCP under real-world packet loss.