Skip to content

VR: route replies to remote access VPN clients through the tunnel - #14548

Open
willbennett-devops wants to merge 1 commit into
apache:4.22from
willbennett-devops:fix-vpn-client-reply-route
Open

willbennett-devops wants to merge 1 commit into
apache:4.22from
willbennett-devops:fix-vpn-client-reply-route

Conversation

@willbennett-devops

Copy link
Copy Markdown

Description

This PR fixes replies from the VPC router to remote access VPN clients leaving by a public NIC's gateway instead of
the client's ppp interface, which makes LB rules and the router's public addresses unreachable over the VPN.

A reply the router sends from a public address (haproxy for an LB rule, an ICMP echo reply) matches
from <public network> lookup Table_ethX, and those tables have no route for the client range. Changes, all in
systemvm/debian/opt/cloud/bin/configure.py:

  • CsRemoteAccessVpn.client_routes() adds throw <client network> table Table_ethN proto 147 for each client
    network (the client range summarized into CIDRs, plus local_ip/32) to every Table_ethN that an ip rule looks
    up. The lookup then continues in main, which holds the per-session ppp routes. This is how tier networks are already
    kept out of these tables. No policy rule is added, so rule order doesn't matter.
  • The routes carry protocol 147 so only routes created here are listed and deleted. It reconciles desired against
    present and runs nothing when they match. If ip rule show returns nothing it leaves the routes alone.
  • It's called from main() on every pass, right after address processing, not from CsRemoteAccessVpn.process(). An
    ip_associations pass can flush and rebuild a per-NIC table without running the iptables executors. A removed VPN
    is already gone from the data bag, so its routes are removed on the next pass.
  • client_networks() leaves out, and logs, a client network that overlaps a subnet on the router, a site-to-site
    peer network, or a static route that equals or lies inside it. CloudStack only checks the range against the VPC
    CIDR, and taking those addresses over would break that traffic.
  • remoteaccessvpn_iptables() adds -A PREROUTING -s <client network> ! -i ppp+ -j DROP in mangle for each client
    network. With the throws in place, a packet arriving on a public NIC with a client's address as source would get its
    reply sent into that client's tunnel; this drops it. The rules are written in iptables-save form so CsNetfilter
    finds them on the next pass and removes them with the VPN.

Fixes: #14547

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • Build/CI
  • Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

How Has This Been Tested?

Unit tests: new systemvm/test/TestCsRemoteAccessVpnClientRoutes.py runs client_routes() against a fake
ip rule/ip route with CsHelper.execute mocked: throws in every per-NIC table and none on a repeated pass, a
flushed table and a new NIC table get theirs back, VPN removed or create: false, client range changed, ip rule show
empty, an unreadable entry, overlaps with a router subnet, a site-to-site peer and a static route (and a containing
route, a revoked route and a removed peer that don't count), routes it doesn't own left alone, and the spoofing rules.

On a 4.23.0.0 KVM lab (the changed code is identical in 4.22 and main), VPC with redundant routers, both routers
patched, an L2TP/IPsec client connected:

  • LB rule over the VPN: HTTP 200 from the LB address, ping to the router's public addresses answered (both time out
    without the change). Port forwards still work.
  • LB and ACL rule create and delete with the session up: no route or rule changes on those passes.
  • Failover and failback: the backup router has the routes before it becomes primary; LB works through each router.
  • VPN removed with the backup router stopped, then the backup started: routes and drop rules removed on the primary,
    nothing stale on the backup. VPN created again: routes back.
  • Router stop and start, and reboot, through the API: the routes and drop rules are back once the router is
    configured.
  • Spoofed source: forged ICMP echoes with a connected client's address sent to the router's public address from a
    host on that public network are dropped (counter matches), no reply goes into the client's tunnel. With the drop
    rule taken out, the replies went into the tunnel.
  • Restart of the VPC with cleanup, then the change applied to the new routers: same results for LB, churn and spoofing.

Failover, failback, VPN removal and the reboots were run on an earlier revision that put the spoofing rules in a
separate chain; the final version was checked again for LB, rule churn and spoofing on both routers.

How did you try to break this feature and the system with this change?

Overlaps (unit tests, not on the lab): a site-to-site peer network that overlaps the client range turns the routes
off for that network rather than dropping the peer's traffic, and table 220 and its rule are never touched. A static
route through a private gateway that contains the client range keeps the routes, since the ppp host route in main is
more specific. Client ranges that overlap a public or tier subnet are logged and skipped. The spoofing rules leave IPsec itself alone: the outer packets
carry the client's public address, and only the decapsulated PPP payload, which arrives on pppN, has a client range
source. The pool must not be used by hosts reached through a private gateway or static route; their packets would
arrive on that NIC and be dropped.

On a router with more than one public NIC, anything the router answers
itself from an address on a non-source-NAT public NIC (an LB address,
ICMP) matched "from <public subnet> lookup Table_ethN" and left through
that NIC's gateway instead of the client's ppp interface, so VPN
clients could not reach a load balancer on such an address.

Add a throw route for each VPN client network to every Table_ethN, so
the lookup falls through to main and its ppp host routes. The routes
carry their own protocol number and are reconciled on every pass from
the active VPN entries, so they follow NIC changes and VPN deletion.
Client networks that overlap a router subnet, a static route or a
site-to-site peer network are skipped. Packets from a client network
that arrive on any interface other than ppp are dropped, so a spoofed
source can't have the reply sent into a client's tunnel.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant