strange route problem

From: Niki Denev <nike_d_at_cytexbg.com>
Date: Sun, 03 Jul 2005 23:14:15 +0300
Hello,

I have a very strange routing problem with one machine acting
as a gateway for a small home network.
The machine has 3 ethernet interfaces two of them connected to two
different ISPs and the third to the home network.
The first ethernet adapter is connected to the first ISP with fixed IP
address and also the default route on the machine is set to this ISP.
The second eth. adapter is connected to the second ISP which allocates
IPs dynamicaly with DHCP, so i use 'supersede routers $first_isp_gw' in
dhclient.conf to prevent the default route changing.
I wanted to use the second ISP only for selected networks that are 
faster through it so i installed several static routes for them.
And after a while the dhcp lease to the second ISP renews, there are 
messages in the logs about the inteface going DOWN and UP and then the 
routing for these networs stops working.
Whats really strange is that if i traceroute IP from on of these 
statically routed networks i can clearly see that the packets are 
traveling via the default gw on the first ISP, while there is route for 
them to go via the second ISP.
"netstat -r" shows the static routes, "route get $ip" again shows them 
correcly and in UP state, but they are not used?
Here is a short example :

ste0 with ip 10.0.0.1 is the internal interface
ste1 with ip 192.168.0.152 is the fixed ip to the first ISP
ste2 is running DHCP with the second ISP and has ip 192.168.100.57

root_at_mars# netstat -rnfinet

Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            192.168.0.129      UGS         2  2788492   ste1
10/28              link#1             UC          0        0   ste0
192.168.0.128/27   link#2             UC          0        0   ste1
192.168.0.129      xx:xx:xx:xx:xx:xx  UHLW        1        0   ste1   1076
127.0.0.1          127.0.0.1          UH          1        3    lo0
192.168.100/19     link#3             UC          0        0   ste2
192.168.100.1      xx:xx:xx:xx:xx:xx  UHLW        1        0   ste2    249
192.168.100.57     127.0.0.1          UGHS        0        0    lo0
10.10.10           192.168.100.1      UGS         0     1016   ste1

root_at_mars# route -n get 10.10.10.1

    route to: 10.10.10.1
destination: 10.10.10.0
        mask: 255.255.255.0
     gateway: 192.168.100.1
   interface: ste1
       flags: <UP,GATEWAY,DONE,STATIC>
  recvpipe  sendpipe  ssthresh  rtt,msec    rttvar  hopcount      mtu 
   expire
        0         0         0         0         0         0      1500 
       0

root_at_mars# traceroute -n 10.10.10.1
traceroute to 10.10.10.1 (10.10.10.1), 64 hops max, 40 byte packets
  1  192.168.0.129  6.514 ms  4.047 ms  7.537 ms
  (snipped traceroute continuing to destination via wrong route)

The machine runs stripped down install of recent 6.0-CURRENT,
with read only mounted root filesystem on a 128MB CF card and
mfs /var and /tmp.

I'm also using pf(4) for filtering and nat on the both external interfaces.
No policy routing is done, just two "reply-to"s on the external 
interfaces so anything that comes in one of them is routed via
the same interface in the oposite direction.

P.S.: manually doing route change -net 10.10.10.0/24 192.168.100.1 fixes 
the problem. (until probably the next dhcp renew if thats what is 
causing it) As if the dhclient caused interface UP/DOWN change makes the 
routes inactive?

Thanks for any suggestions in advance!

--niki
Received on Sun Jul 03 2005 - 18:13:52 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:38:38 UTC