Issues with Jails/Routes/FIBs

From: Kevin Mai <kma_at_mrecic.gov.ar>
Date: Thu, 25 Nov 2010 17:38:58 -0200 (ARST)
Hi folks! 

I'm facing an issue here while trying to define separate routing tables for each jail and host. 

Let me show you briefly how it's done: 

The server has 3 physical NICs, each one connected to a different network (say, public network A, public network B, and LAN). 

Currently, the default gateway is set to be the LAN gateway, even though the two jails can see their own public network subnet. 

Internet: 
Destination Gateway Flags Refs Use Netif Expire 
default 172.16.2.1 UGS 1 3935 bce2 
127.0.0.1 link#5 UH 0 0 lo0 
172.16.2.0/24 link#3 U 0 0 bce2 
172.16.2.127 link#3 UHS 0 0 lo0 
100.16.97.0/24 link#1 U 0 0 bce0 
100.16.97.5 link#1 UHS 0 0 lo0 
100.16.98.0/24 link#2 U 0 0 bce1 
100.16.98.5 link#2 UHS 0 0 lo0 

100.16.97.0/24 and 100.16.98.0/24 are the two public networks and 172.16.2.0/24 is the LAN. 

I have already tried removing devfs rules from the jails, setting securelevel to -1 but I'm still out of luck.. 

I know setfib can define alternate routing tables, and I even created a default gateway for two fibs, 1 & 2: 

[root_at_mrefns09 ~]# setfib 2 netstat -rn 
Routing tables 

Internet: 
Destination Gateway Flags Refs Use Netif Expire 
default 100.16.98.100 UGS 14 906 bce1 
127.0.0.1 link#5 UH 0 0 lo0 
172.16.2.0/24 link#3 U 0 0 bce2 
100.16.97.0/24 link#1 U 0 39 bce0 
100.16.98.0/24 link#2 U 0 0 bce1 

[root_at_mrefns09 ~]# setfib 1 netstat -rn 
Routing tables 

Internet: 
Destination Gateway Flags Refs Use Netif Expire 
default 100.16.97.100 UGS 0 1758 bce0 
127.0.0.1 link#5 UH 0 0 lo0 
172.16.2.0/24 link#3 U 0 0 bce2 
100.16.97.0/24 link#1 U 0 44 bce0 
100.16.98.0/24 link#2 U 0 4 bce1 

And i've added the proper settings in rc.conf.. 

jail_athea97_ip="100.16.97.5 netmask 255.255.255.0" 
jail_athea97_fib=1 


jail_athea98_ip="100.16.98.5 netmask 255.255.255.0" 
jail_athea98_fib=2 

Am I missing something? because once I get into the jail the routing table is the same: 

[root_at_athea97 /]# netstat -rn 
Routing tables 

Internet: 
Destination Gateway Flags Refs Use Netif Expire 
default 172.16.2.1 UGS 13 6175 bce2 
127.0.0.1 link#5 UH 0 0 lo0 
172.16.2.0/24 link#3 U 0 0 bce2 
172.16.2.127 link#3 UHS 0 0 lo0 
100.16.97.0/24 link#1 U 0 0 bce0 
100.16.97.5 link#1 UHS 0 0 lo0 
100.16.98.0/24 link#2 U 0 0 bce1 
100.16.98.5 link#2 UHS 0 0 lo0 

[root_at_athea97 /]# setfib 1 netstat -rn 
Routing tables 

Internet: 
Destination Gateway Flags Refs Use Netif Expire 
default 100.16.97.100 UGS 15 1814 bce0 
127.0.0.1 link#5 UH 0 0 lo0 
172.16.2.0/24 link#3 U 0 0 bce2 
100.16.97.0/24 link#1 U 0 44 bce0 
100.16.98.0/24 link#2 U 0 4 bce1 

The other jail is acting the same way. I know that since I'm doing a jexec, the shell will have the host's route because, but, how can I know if it's getting the alternate routing table? 

Thanks, 

Kevin 
Received on Thu Nov 25 2010 - 19:08:48 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:09 UTC