AllGoodBits.org

Scenario

• A virtualization host with several guests
• Several network interfaces on different networks
• A bridged setup so that guests can have ip addresses on the outside network
• Some guests on each network

Problem

• Each guest's traffic should be routed to/from the correct network.
• Old fashioned linux networking (net-tools) has difficulty providing more than one gateway, but I need to provide a gateway for each network

Solution

• Using iproute2 we can manage multiple routing tables and apply rules such that traffic to/from particular addresses use the appropriate routing table, configured with the right gateway.

Here's what we're going to end up with:

eth0/br0

ip addr: 192.168.1.128 network: 192.168.1.0/24 gw 192.168.1.1

eth1/br1

ip addr: 10.1.0.10 network: 10.1.0.0/24 gw 10.1.0.1

Here's what we start with. It's a basic result of having configured a single interface to bridge:

#ip route show
192.168.1.0/24 dev br0
default via 192.168.1.1 dev br0

Create a custom routing table:

echo "1   myorg" >> /etc/iproute2/rt_tables

Specify a static route to the secondary gateway (this is necessary if and only if it's on the same ethernet segment):

ip route add 10.1.0.1 scope link dev br1

Specify the conditions that should use our custom table:

ip rule add from 10.1.0.0/24 table myorg

Teach our custom routing table the gateway it should use:

ip route add default via 10.1.0.1 dev br1 table myorg

Scenario 2

• A host has an address on one network in the normal way
• It has another NIC that is connected to a port on another network
• Traffic for each network should go to the correct gateway (and through the correct NIC)
• One should be considered the default
• eth0 is the primary NIC: 10.41.0.10/22
• eth1 is the secondary NIC: 10.8.0.79/22
• The main difference between this and the previous scenario is that the secondary network is used by this host, not handed to a bridge for the use of kvm guests.

Create a custom routing table:

echo "1   myorg" >> /etc/iproute2/rt_tables

Specify a static route to the secondary gateway (this is necessary if and only if it's on the same ethernet segment):

ip route add 10.8.0.1 scope link dev eth1

Specify the conditions that should use our custom table:

ip rule add from 10.8.0.79 table myorg

Teach our custom routing table the gateway it should use:

ip route add default via 10.8.0.1 dev eth1 table myorg

References

Again the most important reference link for me is the Linux Advanced Routing & Traffic Control HOWTO, but also IPROUTE2 Utility Suite Howto and Linux Advanced Routing Mini HOWTO.