In my last post, I covered how to route packages from a specific VLAN through a VPN on the USG. Here, I will show how to use policy-based routing on Linux to route packets from specific processes or subnets through a VPN connection on a Linux host in your LAN instead. You could then point to this host as the next-hop for a VLAN on your USG to achieve the same effect as in my last post.

Note that this post will assume a modern tooling including firewalld and NetworkManager, and that subnet 192.168.10.0/24 is your LAN. This post will send packets coming from 192.168.20.0/24 to VPN, but you could customize that as you see fit (e.g. send specific only hosts from your normal LAN subnet instead).

VPN network interface setup

First, let’s create a VPN firewalld zone so we can easily apply firewall rules just to the VPN connection:

firewall-cmd --permanent --new-zone=VPN
firewall-cmd --reload

Next, create the VPN interface with NetworkManager:

VPN_USER=openvpn_username
VPN_PASSWORD=openvpn_password

# Setup VPN connection with NetworkManager
dnf install -y NetworkManager-openvpn
nmcli c add type vpn ifname vpn con-name vpn vpn-type openvpn
nmcli c mod vpn connection.zone "VPN"
nmcli c mod vpn connection.autoconnect "yes"
nmcli c mod vpn ipv4.method "auto"
nmcli c mod vpn ipv6.method "auto"

# Ensure it is never set as default route, nor listen to its DNS settings
# (doing so would push the VPN DNS for all lookups)
nmcli c mod vpn ipv4.never-default "yes"
nmcli c mod vpn ipv4.ignore-auto-dns on
nmcli c mod vpn ipv6.never-default "yes"
nmcli c mod vpn ipv6.ignore-auto-dns on

# Set configuration options
nmcli c mod vpn vpn.data "comp-lzo = adaptive, ca = /etc/openvpn/keys/vpn-ca.crt, password-flags = 0, connection-type = password, remote = remote.vpnhost.tld, username = $VPN_USER, reneg-seconds = 0"

# Configure VPN secrets for passwordless start
cat > /etc/NetworkManager/system-connections/vpn

[vpn-secrets]
password=$VPN_PASSWORD
EOF
systemctl restart NetworkManager

Configure routing table and policy-based routing

Normally, a host has a single routing table and therefore only 1 default gateway. Static routes can be configured for next-hops, this is configuring the system to route based a packet’s destination address, and we want to know how route based on the source address of a packet. For this, we need multiple routing tables (one for normal traffic, another for VPN traffic) and Policy Based Routing (PBR) to define rules on how to select the right one.

First, let’s create a second routing table for VPN connections:

cat > /etc/iproute2/rt_tables
100 vpn
EOF

Next, setup an IP rule to select between routing tables for incoming packets based on their source addres:

# Replace this with your LAN interface
IFACE=eno1

# Route incoming packets on VPN subnet towards VPN interface
cat > /etc/sysconfig/network-scripts/rule-$IFACE
from 192.168.20.0/24 table vpn
EOF

Now that we can properly select which routing table to use, we need to configure routes on the vpn routing table:

cat  /etc/sysconfig/network-scripts/route-$IFACE
# Always allow LAN connectivity
192.168.10.0/24 dev $IFACE scope link metric 98 table vpn
192.168.20.0/24 dev $IFACE scope link metric 99 table vpn

# Blackhole by default to avoid privacy leaks if VPN disconnects
blackhole 0.0.0.0/0 metric 100 table vpn
EOF

You’ll note that nowhere do we actually define the default gateway - because we can’t yet. VPN connections often dynamically allocate IPs, so we’ll need to configure the default route for the VPN table to match that particular IP each time we start the VPN connection (we’ll do so with a smaller metric figure than the blackhole above of 100, thereby avoiding the blackhole rule).

So, we will configure NetworkManager to trigger a script upon bringing up the VPN interface:

cat  /etc/NetworkManager/dispatcher.d/90-vpn
VPN_UUID="\$(nmcli con show vpn | grep uuid | tr -s ' ' | cut -d' ' -f2)"
INTERFACE="\$1"
ACTION="\$2"

if [ "\$CONNECTION_UUID" == "\$VPN_UUID" ];then
  /usr/local/bin/configure_vpn_routes "\$INTERFACE" "\$ACTION"
fi
EOF

In that script, we will read the IP address of the VPN interface and install it as the default route. When the VPN is deactivated, we’ll do the opposite and cleanup the route we added:

cat  /usr/local/bin/configure_vpn_routes
#!/bin/bash
# Configures a secondary routing table for use with VPN interface

interface=\$1
action=\$2

tables=/etc/iproute2/rt_tables
vpn_table=vpn
zone="\$(nmcli -t --fields connection.zone c show vpn | cut -d':' -f2)"

clear_vpn_routes() {
  table=$1
  /sbin/ip route show via 192.168/16 table \$table | while read route;do
    /sbin/ip route delete \$route table \$table
  done
}

clear_vpn_rules() {
  keep=\$(ip rule show from 192.168/16)
  /sbin/ip rule show from 192.168/16 | while read line;do
    rule="\$(echo \$line | cut -d':' -f2-)"
    (echo "\$keep" | grep -q "\$rule") && continue
    /sbin/ip rule delete \$rule
  done
}

if [ "\$action" = "vpn-up" ];then
  ip="\$(/sbin/ip route get 8.8.8.8 oif \$interface | head -n 1 | cut -d' ' -f5)"

  # Modify default route
  clear_vpn_routes \$vpn_table
  /sbin/ip route add default via \$ip dev \$interface table \$vpn_table

elif [ "\$action" = "vpn-down" ];then
  # Remove VPN routes
  clear_vpn_routes \$vpn_table
fi
EOF
chmod 755 /usr/local/bin/configure_vpn_routes

Bring up the VPN interface:

nmcli c up vpn

That’s all, enjoy!

Sending all packets from a user through the VPN

I find this technique particularly versatile as one can also easily force all traffic from a particular user through the VPN tunnel:

# Replace this with your LAN interface
IFACE=eno1

# Username (or UID) of user who's traffic to send over VPN
USERNAME=foo

# Send any marked packets using VPN routing table
cat > /etc/sysconfig/network-scripts/rule-$IFACE
fwmark 0x50 table vpn
EOF

# Mark all packets originating from processes owned by this user
firewall-cmd --permanent --direct --add-rule ipv4 mangle OUTPUT 0 -m owner --uid-owner $USERNAME -j MARK --set-mark 0x50
# Enable masquerade on the VPN zone (enables IP forwarding between interfaces)
firewall-cmd --permanent --add-masquerade --zone=VPN

firewall-cmd --reload

Note 0x50 is arbitrary, as long as it the rule and firewall rule match, you’re fine.