Simple policy based routing in practice

Sometimes the network setup isn't the one that you find in a textbook. Policy based routing is a mechanism to choose a route based on a policy (which can be anything, from the current time to the kind of packet/frame).

A little on the environment...

For a lab demo I installed OpenShift in a server under NAT, without the capability of port forwarding. In order to give the server an external access, I made a VPN with a VPS that we had in place for some projects.

Unfortunately the simple port forwarding wasn't enough due to the fact that the VPS had some services running on the same port of OpenShift (ie. HTTPs). So, another IP address was prepared and assigned to the VPS.

All right. We'll configure the VPS to forward the traffic for 203.0.113.5 to 192.0.2.2 (via the VPN). That's it? No. In fact, the Router here will do an asymmetric routing: while an incoming packet will use the path (internet)->VPS->(VPN)->Router->OpenShift, the response will go directly OpenShift->Router->(internet) as the Router, by default, is looking to the routing table without "memory" (ie. the fact that the incoming packet was received by a different interface doesn't have an impact in any future routing decision, by default). The asymmetric routing isn't bad per-se: in this case, the NAT is interfering and it's breaking any connection (as the reply public address is different than the VPS public address used for OpenShift).

The goal is to make a policy that says: ok, use a connection tracking module and, for each connection that pass through VPS, forward replies back through VPS. That is policy based routing (PBR).

Note: there might be an improvement on the load (and routing speed) by avoiding the connection tracking module. I configured the Router to use conntrack because this is a lab experiment, so speed is not a priority.

Let's configure!

PRE: I assume that the VPS and the Router are linked with a VPN and the routing is correctly configured. How to setup the VPN is not the goal of this post, so I'll not show how to do that.

We need to configure the Router and the VPS in order to make this works as expected. The Router is running MikroTik RouterOS (but PBR can be configured in many mid- and high-end routers), and the VPS is using Debian GNU/Linux. In the VPS, a simple IP/port forwarding is sufficient:

# iptables -t nat -I PREROUTING -d 203.0.113.5 -j DNAT --to-destination 192.0.2.2
# iptables -t nat -I POSTROUTING -s 192.0.2.2 -o wan_interface0 -j SNAT --to-source 203.0.113.5
# iptables -I FORWARD -i wan_interface0 -d 192.0.2.2 -p tcp --dport 443 -j ACCEPT

This is sufficient in order to forward the traffic for the HTTPs port to OpenShift, and do the address translation back on replies.

Now, on the MikroTik Router we need to create a new routing table (I named it "via-VPS") with the link addresses for both the VPN and the LAN where OpenShift is, and the default route via the VPN address of VPS:

# /ip route
# add distance=1 gateway=192.0.0.2 routing-mark=via-VPS
# add distance=1 dst-address=192.0.0.0/24 gateway=vpn_interface0 pref-src=192.0.0.1 routing-mark=via-VPS scope=10
# add distance=1 dst-address=192.0.2.0/24 gateway=lan_interface0 pref-src=192.0.2.1 routing-mark=via-VPS scope=10

Then, configure the firewall and the conntrack in order to track each connection coming from the VPN and make next routing decisions (on replies) using the new routing table:

# /ip firewall
# add action=mark-connection chain=prerouting connection-state=new dst-address=192.0.2.2 dst-port=443 in-interface=vpn_interface0 new-connection-mark=via-VPS passthrough=yes protocol=tcp
# add action=mark-routing chain=prerouting connection-mark=via-VPS new-routing-mark=via-VPS passthrough=yes