Unified Cloud Routing: Building an Anycast-to-Unicast WireGuard Multi-Cloud Ingress Overlay

Modern multi-cloud deployments face a frustrating paradox. The same architectural decision that buys resilience — spreading workloads across AWS, GCP, and Azure — also fragments your network into three incompatible private addressing schemes, three routing domains, and three sets of cloud-specific transit tools. The default answer from each vendor (Direct Connect, ExpressRoute, Cloud Interconnect) is expensive, rigid, and deepens the very lock-in you were trying to avoid.

A better architecture treats the public internet as a dumb on-ramp and uses a BGP Anycast edge combined with an automated WireGuard overlay mesh as the private backbone that ties all three clouds into a single, routable flat network. This article walks through how that architecture works in practice — the routing mechanics, the dynamic orchestration layer, the MTU and asymmetric routing traps, and the security hardening required before you expose a globally reachable IP that tunnels directly into your entire multi-cloud footprint.

---

The Two-Tier Model

The architecture divides cleanly into a public tier and a private tier.

Tier 1: BGP Anycast at the Edge

Anycast is a network addressing scheme in which a single IP prefix is simultaneously announced from multiple physical locations. When a client sends a packet to that IP, the BGP path-selection process running across the public internet — primarily shortest AS-path length — routes the packet to whichever Point of Presence (PoP) is topologically closest to that client.

              [ Global Client ]
                     |
         ( Internet / BGP routing )
                     |
      +--------------+--------------+
      |  Anycast prefix (same IP)   |
      v                             v
+------------------+       +------------------+
| PoP 1 – Europe   |       | PoP 2 – US-East  |
| Edge proxy /     |       | Edge proxy /     |
| WireGuard node   |       | WireGuard node   |
+------------------+       +------------------+

The result is geographic proximity-matching without DNS manipulation: a client in Frankfurt reaches the European PoP, a client in Virginia reaches US-East, and neither client needs to be told which PoP to use. Traffic enters the provider-controlled network at the first opportunity, minimising exposure to unpredictable public-internet paths before it reaches your infrastructure.

Anycast is already the backbone of large-scale internet infrastructure. DNS resolvers like Cloudflare’s 1.1.1.1 and Google’s 8.8.8.8 rely on exactly this mechanism to serve billions of queries globally with sub-millisecond geographic steering.

Tier 2: The WireGuard Overlay Mesh

Once a packet lands at an Anycast edge node, it leaves the public internet entirely. The edge node must now forward that packet to the correct backend service — which may live in AWS us-east-1, GCP asia-southeast1, or Azure West US — without bouncing back onto the public internet or paying for proprietary cloud interconnect circuits.

This is where the WireGuard overlay mesh takes over. Every edge node and every backend node across all three clouds participates in an encrypted UDP mesh. Traffic is encapsulated into a WireGuard tunnel and delivered directly across the mesh to the target unicast IP inside the appropriate cloud VPC.

The three cloud networks cease to be three separate domains. From the perspective of the routing daemon running on each node, the entire multi-cloud footprint is a single flat private network reachable over encrypted WireGuard interfaces.

---

WireGuard: Kernel-Native Performance, Fixed Cryptography

WireGuard was merged into the Linux kernel mainline in kernel 5.6, released in March 2020, and has shipped as a built-in module in every Linux kernel since. For deployments on older kernels or non-Linux platforms, userspace implementations like wireguard-go are available, as are eBPF-based data planes that move the hot path closer to the kernel.

The protocol’s cryptographic suite is intentionally fixed rather than negotiable, eliminating the class of downgrade attacks that plague configurable-cipher protocols like IPsec:

• Curve25519 for Elliptic Curve Diffie-Hellman key exchange
• ChaCha20-Poly1305 (RFC 7539) for authenticated encryption of data packets
• BLAKE2s for hashing and keyed hashing
• SipHash24 for hashtable keys
• HKDF for key derivation

The entire protocol implementation sits in approximately 4,000 lines of code — a deliberate contrast to OpenVPN’s 100,000+ lines — making it auditable by small teams. A formal analysis using CryptoVerif’s calculus of cryptographic games has verified mutual authentication, IND-CCA session-key secrecy, forward secrecy, and post-compromise security across unlimited parallel sessions.

WireGuard re-keys every 120 seconds (or every 2⁶⁰ messages) to provide perfect forward secrecy independent of traffic volume.

---

MTU Arithmetic: Getting the Numbers Right

This is the section where most WireGuard deployments go wrong. WireGuard encapsulates every packet with an outer header stack, and if you do not account for the overhead correctly, the resulting over-sized packets are silently dropped by intermediate transit networks, producing mysterious connection freezes and broken TLS handshakes.

The Actual Overhead Breakdown

For an IPv4 WireGuard deployment, the outer envelope adds:

Layer	Size
Outer IPv4 header	20 bytes
UDP header	8 bytes
WireGuard message header (type, key, nonce)	32 bytes
Poly1305 authentication tag	16 bytes
Total overhead	76 bytes

For IPv6 transport, the outer IPv6 header is 40 bytes instead of 20, bringing the total to 96 bytes.

Given a standard Ethernet MTU of 1500 bytes, the safe WireGuard interface MTU is:

• IPv4 transport: 1500 − 76 = 1424 bytes (WireGuard’s practical default rounds to 1420 for compatibility margin)
• IPv6 transport: 1500 − 96 = 1404 bytes (typically rounded down to 1400)

WireGuard’s default MTU of 1420 is chosen to work safely for both IPv4 and IPv6 underlay on a standard 1500-byte Ethernet path.

TCP MSS Clamping

For TCP traffic, the edge proxy must rewrite the Maximum Segment Size field in TCP SYN packets to match the reduced MTU. Inside a WireGuard tunnel with an interface MTU of 1420, the safe MSS values are:

• IPv4 TCP inside the tunnel: 1420 − 20 (inner IP) − 20 (TCP) = 1380 bytes
• IPv6 TCP inside the tunnel: 1420 − 40 (inner IP) − 20 (TCP) = 1360 bytes

The iptables rule to enforce this on the WireGuard interface is:

# Clamp MSS for IPv4 TCP transiting the WireGuard interface
iptables -t mangle -A FORWARD -i wg0 -p tcp \
  --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1380

# Or clamp dynamically to the discovered path MTU
iptables -t mangle -A FORWARD -i wg0 -p tcp \
  --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu

Without clamping, a client that negotiates a standard MSS of 1460 (Ethernet 1500 − 40 header bytes) produces segments that exceed the tunnel MTU. The excess packets are dropped silently, causing exactly the class of intermittent failures — large file transfers stalling, SCP failing while SSH works, partial page loads — that are notoriously difficult to debug.

ICMP Fragmentation Needed

For non-TCP protocols (UDP, ICMP, custom tunnels within the tunnel), MSS clamping does not apply. Path MTU Discovery (PMTUD) must work end-to-end, which requires that ICMP Type 3 Code 4 (“Destination Unreachable — Fragmentation Needed”) packets are not blocked by cloud security groups or internal firewalls. Blocking these ICMP messages creates a PMTUD black hole: the sender never learns the path MTU is insufficient and continues sending over-large packets that are silently dropped.

Cloud security groups default to blocking all ICMP in many configurations. Explicitly allowing ICMP Type 3 from the WireGuard interface ranges is a required step in any production deployment.

---

Dynamic Peer Orchestration

A global WireGuard mesh across hundreds of ephemeral cloud nodes — spinning up and down with autoscaling events across three providers — cannot be managed with static configuration files. The orchestration layer must automate peer discovery, key distribution, and tunnel lifecycle without interrupting active sessions.

Bootstrapping New Nodes

When a new node spins up inside an AWS Auto Scaling group or a GKE cluster, it executes a bootstrapping sequence before accepting traffic:

1. The node generates a local WireGuard keypair (wg genkey | tee privatekey | wg pubkey > publickey).
2. It registers its WireGuard public key and internal unicast IP to a distributed configuration store — typically etcd or Consul — reachable from all three cloud environments.
3. The global orchestration daemon detects the registration event (via etcd watch or Consul event stream) and pushes updated peer configurations to all active mesh members.
4. Tunnels to the new node establish automatically. Because WireGuard is stateless at the session level — it maintains no per-connection state, just cryptographic routing tables — adding peers does not require restarting existing tunnels or dropping active connections.

Open-Source Orchestration Tooling

Several mature projects handle this orchestration layer:

Tailscale wraps WireGuard with a coordination server that manages peer discovery, NAT traversal, key distribution, and device identity. The data plane is WireGuard; the control plane is proprietary (though the clients are open-source). It supports up to 100 devices on the free tier and adds user-friendly ACLs and SSO integration.

NetBird (Apache 2.0 licensed, ~13,000 GitHub stars as of mid-2025) is a fully self-hostable alternative to Tailscale. It provides the complete stack — management server, dashboard, client — and supports OIDC identity providers including Keycloak and Azure AD. For multi-cloud infrastructure where data sovereignty or on-premises control of the coordination plane is required, NetBird is the most commonly deployed choice.

Headscale re-implements Tailscale’s coordination server for self-hosting while retaining compatibility with the official Tailscale clients. It handles peer introduction and NAT traversal without the Tailscale SaaS dependency.

For teams building custom orchestration, the common pattern is a Go daemon that watches the cloud-provider autoscaling APIs and the etcd cluster simultaneously, generating and distributing wg set commands as the mesh membership changes.

---

Internal Routing: iBGP over the WireGuard Overlay

The orchestration layer handles peer discovery and tunnel establishment. A separate question is: once the mesh is up, how does an Anycast edge node decide which backend node to send a given packet to, given that the same microservice may be deployed in multiple regions and clouds simultaneously?

The answer is to run an internal BGP (iBGP) daemon directly on the WireGuard interfaces, treating them as the physical transport for an internal routing protocol.

+-----------------------------------------------------------------------+
|                       WIREGUARD OVERLAY MESH                          |
|                                                                       |
|  +------------------------+          +------------------------+       |
|  |  Anycast Edge Node     |========|  AWS backend node      |       |
|  |  (BIRD / FRR router)   | WG mesh  |  (unicast target)      |       |
|  +------------------------+          +------------------------+       |
|           ^                                      ^                    |
|           |                                      |                    |
|           v                                      v                    |
|  +------------------------+          +------------------------+       |
|  |  GCP backend node      |========|  Azure backend node    |       |
|  |  (unicast target)      | WG mesh  |  (unicast target)      |       |
|  +------------------------+          +------------------------+       |
+-----------------------------------------------------------------------+

Each edge node runs a routing daemon — BIRD or FRRouting (FRR) are the two dominant choices — configured to peer over the WireGuard interface IPs.

BIRD (BIRD Internet Routing Daemon), maintained by CZ.NIC Labs, released version 3.0 in December 2024, introducing cooperative multithreading for large-scale BGP sessions. As of mid-2025, the latest stable release is 3.1.x. BIRD is widely deployed as a route server and route reflector at Internet Exchange Points, where it handles over one million BGP routes per peer on modest EC2 t3 hardware with approximately 1.3 GB RAM for five million total routes.

FRRouting (FRR) is the fork of Quagga that became the de-facto open-source full routing stack for Linux since 2017. It provides a complete routing suite (BGP, OSPF, IS-IS, RIP, PIM) and is the better choice when the mesh nodes also need to participate in more complex multi-protocol routing topologies.

Route Advertisement and Failover in Practice

Each backend node advertises the prefixes it serves via iBGP over its WireGuard interface. The edge node collects these advertisements and builds a routing table across the overlay.

Consider a microservice deployed in both GCP asia-southeast1 and Azure West US. An edge node in Singapore measures 8 ms latency to the GCP instance’s WireGuard interface and 140 ms to Azure West US. The iBGP metric matrix assigns the GCP tunnel as the active next-hop.

When the GCP instance fails its health check (or the WireGuard interface goes down and triggers an interface-down event in the routing daemon), the BGP session over that tunnel drops. The routing daemon withdraws the route and reconverges within seconds, shifting all traffic to the Azure path. No DNS TTL expiry. No sticky session state to invalidate. Failover propagates at routing protocol convergence speed across the mesh.

---

The Asymmetric Routing Problem

Anycast creates a topological hazard: the public internet’s BGP paths are not stable. Transient congestion, carrier route flaps, and maintenance windows can cause two consecutive packets from the same TCP flow to arrive at different Anycast edge nodes — one hitting New York, the next hitting Washington D.C.

If each edge node maintains local TCP connection state, the second packet arrives at an edge node with no record of the connection and is dropped. The TCP session breaks.

Mitigation Approaches

Stateless or shared-state ingress. Anycast edge nodes should not maintain per-connection state locally. Using a stateless L4 load balancer (eBPF-based XDP programs are particularly efficient here) combined with a shared session cache — Redis Cluster or a distributed hash table reachable from all PoPs — allows any edge node to handle any packet from any TCP flow without local state. Cloudflare’s Maglev-inspired consistent hashing approach implements this at scale, where the hash of the 4-tuple (src IP, dst IP, src port, dst port) selects a backend deterministically across all edge nodes.

PROXY Protocol v2 for client IP preservation. When the WireGuard overlay carries the encapsulated payload to the unicast backend, the backend sees the internal WireGuard IP of the edge node as the source address. To preserve the original client IP for logging, rate limiting, and geo-blocking, the edge proxy must prepend a PROXY Protocol v2 header to the TCP byte stream before forwarding over the tunnel. The backend application is then configured to read client IP from the PROXY header rather than the network layer.

Connection tracking via eBPF. An alternative to shared external state is to attach eBPF XDP programs to the ingress interface of each Anycast node that compute a consistent hash of the flow 4-tuple and forward mismatched packets directly to the correct sibling edge node via the WireGuard mesh — without breaking the session or involving the application tier.

---

Security Hardening

A globally exposed Anycast IP that connects directly into an internal WireGuard mesh spanning three cloud environments is a high-value target. A compromised edge node without additional controls could offer lateral movement into every backend subnet across AWS, GCP, and Azure.

WireGuard Cryptographic Routing

WireGuard’s foundational security property is cryptographic routing: every peer has a static public key, and the tunnel enforces a strict 1:1 mapping between a peer’s allowed internal IP prefixes and its authenticated key. Any packet arriving over a WireGuard interface whose inner source IP does not match the authenticated key configuration for that peer is silently dropped at the kernel level, before any routing decision is made. You cannot spoof your WireGuard IP without the corresponding private key.

eBPF Micro-Segmentation

Cryptographic routing prevents address spoofing but does not limit what an authenticated edge node can reach once its tunnel is established. Attaching eBPF programs to the WireGuard interfaces (wg0) enforces Layer 3 and Layer 4 access control policies at the kernel level, before packets reach any userspace process.

A typical policy set:

• Edge nodes: permitted to reach only the load-balancer VIPs and health-check endpoints in each cloud VPC. Blocked from reaching database subnets, management planes, and cloud metadata services (169.254.169.254, etc.).
• Backend nodes: permitted to respond to connections initiated from edge WireGuard IPs only. Blocked from initiating connections across cloud boundaries without explicit policy.

eBPF enforcement runs at near-zero latency compared to userspace firewall daemons, and because it executes inside trusted kernel space, it remains enforced even if the edge node’s userspace processes are fully compromised.

Tools like Cilium (which combines Kubernetes NetworkPolicy, WireGuard encryption, and eBPF-based micro-segmentation) make this pattern accessible without writing raw eBPF programs. Cilium’s Hubble observability layer provides per-flow network telemetry across the mesh without requiring a sidecar on every pod.

Preshared Keys and Post-Quantum Posture

WireGuard optionally supports a 256-bit symmetric preshared key (PSK) per peer pair, mixed into the HKDF key derivation during the handshake. The PSK provides a defence-in-depth layer: if the Curve25519 key exchange is compromised (for example by a “harvest now, decrypt later” quantum adversary recording today’s handshakes), the PSK prevents decryption because recovering the session keys also requires the PSK.

Critical nuance: the PSK hardens against quantum attacks but does not eliminate the risk entirely. Curve25519 key exchange remains vulnerable to Shor’s algorithm on a cryptographically relevant quantum computer. A PSK-protected session is more resistant than one without, but for long-term security guarantees on persistent mesh sessions, the emerging approach is a hybrid key exchange — combining classical Curve25519 with a post-quantum Key Encapsulation Mechanism (KEM) such as ML-KEM (NIST FIPS 203, formerly Kyber). Projects like OQS-WireGuard implement this hybrid scheme without modifying the core WireGuard protocol, instead delivering a post-quantum PSK over a separate ML-KEM hybrid TLS 1.3 channel before WireGuard sessions establish.

PSK rotation should be automated. A 30-day maximum PSK lifetime, distributed via the same etcd/Consul store used for peer configuration, provides adequate operational security without requiring manual intervention.

---

Observability Across the Mesh

One of the underappreciated benefits of this architecture is the uniform observability surface it creates. Traditional multi-cloud monitoring requires piecing together CloudWatch (AWS), Cloud Monitoring (GCP), and Azure Monitor — each with different data models, different latencies, and different query languages.

With WireGuard interfaces as the uniform data plane, all network telemetry is available through standard Linux tooling regardless of which cloud the node runs in:

# Per-peer transfer counters (updated every 30 seconds by default)
wg show wg0

# Interface-level packet and byte counters
ip -s link show wg0

# BPF-based per-flow telemetry (if Cilium/Hubble is deployed)
hubble observe --follow --namespace production

Exporting WireGuard interface metrics via a Prometheus node_exporter textfile collector and feeding them into a centralised Grafana dashboard gives network engineers a single pane of glass for packet loss, latency, and throughput across every tunnel in the mesh — AWS to GCP, GCP to Azure, edge to backend — without cloud-specific connectors.

For latency-aware routing decisions, the BIRD or FRR daemon can be configured to consume these metrics and adjust BGP route preferences dynamically, closing the loop between observability and traffic steering.

---

Trade-offs and When This Architecture Does Not Apply

This is a complex architecture with real operational costs. Before adopting it, the trade-offs are worth stating plainly.

Operational complexity is high. Running BGP daemons, WireGuard mesh orchestration, eBPF policies, and shared session state across three cloud providers requires deep expertise in Linux networking. Teams without this background will spend significant time debugging issues that a simpler architecture (cloud-native load balancers per region, with DNS-based routing) would not encounter.

WireGuard is UDP-only. The tunnel uses UDP as its transport. Cloud environments that perform aggressive UDP shaping, or networks with broken PMTUD, require careful MTU tuning and testing. TCP-based tunnels (like those in some ZTNA products) tolerate these environments more gracefully, at the cost of the TCP-over-TCP performance penalty.

BGP Anycast does not guarantee nearest-exit routing. AS-path length is not the same as geographic proximity. A client in Southeast Asia may be steered to a PoP in Japan rather than Singapore if the Japanese PoP’s ASN has fewer hops in the BGP path from that client’s ISP. Traffic engineering (via BGP communities, AS-path prepending, or localpref manipulation) is required to tune steering for specific ISP relationships.

The asymmetric routing problem never fully goes away. You can mitigate it with stateless ingress and consistent hashing, but the underlying cause — BGP path instability on the public internet — is outside your control. Deployments that cannot tolerate any TCP session disruption should evaluate whether Anycast is the right ingress mechanism, or whether they need a DNS-based geographic steering approach with longer TTLs and health-checked failover instead.

---

Conclusion

The Anycast-to-Unicast WireGuard multi-cloud ingress overlay achieves something that proprietary cloud transit tools cannot: a genuinely cloud-agnostic network backbone where migrating a workload from AWS to GCP requires only spinning up a new WireGuard peer and announcing its route into the iBGP mesh.

The architecture’s strengths are well-defined. BGP Anycast provides geographic client steering without DNS manipulation. The WireGuard mesh provides encrypted, kernel-native connectivity between clouds without the cost or rigidity of Direct Connect or ExpressRoute. Dynamic iBGP over the overlay provides sub-second failover based on real-time routing daemon health checks. eBPF at the ingress enforces least-privilege network policy below the application layer.

Its costs are equally well-defined: operational complexity, the need for Linux networking expertise at every layer, and the inherent limitations of BGP as a steering mechanism.

For engineering teams who have outgrown per-cloud native load balancers and need a unified, observable, vendor-neutral network backbone that scales across regions and providers, this architecture represents a sound — if demanding — blueprint.

---

Changelog

• Draft v1 → v1.1: Corrected WireGuard encapsulation overhead from 60 bytes to 76 bytes (IPv4) / 96 bytes (IPv6), accounting for the 16-byte Poly1305 authentication tag that the original draft omitted. Updated recommended WireGuard interface MTU to 1420 (default) and corrected MSS clamping value to 1380 for IPv4 TCP. Added kernel mainline date (Linux 5.6, March 2020). Expanded cryptographic suite to include BLAKE2s, SipHash24, and HKDF per the WireGuard whitepaper. Corrected PSK post-quantum claim: PSK hardens but does not eliminate quantum risk; added hybrid ML-KEM approach. Added BIRD 3.0 release (December 2024) and multithreaded architecture note. Replaced “Kilo” (unmaintained) with NetBird as the current open-source orchestration reference. Added trade-offs section. Removed all unverified performance statistics.