ezjails, systat -ifstat, and multiple network cards

From: Preston Hagar <prestonh_at_gmail.com>
Date: Thu, 13 Feb 2014 12:59:14 -0600
I have a server setup with FreeBSD-10.0-RELEASE.  It has 3 Intel gigabit
network cards in it, em0, em1, and em2.  I have multiple ezjails setup that
run various things.

One jail, called db, runs a postgresql database.  It was my intention to
give it em0 all to itself.   The other jails and host machine should be
going through em2.  em1 currently isn't being used.

If I do an ifconfig, I see that em0 has the alias IP for my db jail and em2
has the alias IP for all other jails.  All the jails respond to network
traffic as expected and seemingly work fine.

The weird thing is when I do a systat -ifstat from the host, it should
essentially all traffic going through em0.  Some of the jails that run off
of em2 (as defined in their jail config files and seen in ifconfig) have
large data transfers and/or are web servers with lots of photos.  I have
even tried to manually scp a large file out of a jail setup through em2 and
the numbers don't seem to budge.

If I do netstat -i -b -n -I  and check em0 and em2, it seems to support the
numbers shown by systat -ifstat.  However, if I use trafshow or iftop (both
of which require choosing one interface at a time), they both seem to
indicate the traffic flowing through the interfaces as I would expect.

So I was curious if anyone had seen something like this before or had any
ideas of what is going on.  I have net.fibs=2 set in /boot/loader.conf, but
in all the jails I current have jail_name_fib="" as I haven't got around to
fullying setting up fibs.  Is that perhaps the issue?  Is there any way to
determine with certainty which jail is using which interface short of
physically pulling a network cable and seeing what stops working?

Here are the relevant lines from my db (the one that should be on em0)
config:

export jail_db_hostname="db"
export jail_db_ip="em0|10.1.10.2"

>From another jail on em2 called www:

export jail_www_hostname="www"
export jail_www_ip="em2|10.1.10.7"

from ifconfig

em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
ether 08:60:6e:13:94:06
inet 10.1.1.4 netmask 0xffff0000 broadcast 10.1.255.255
inet6 fe80::a60:6eff:fe13:9406%em0 prefixlen 64 scopeid 0x1
inet 10.1.10.2 netmask 0xffffffff broadcast 10.1.10.2
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active

em2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
ether 68:05:ca:13:74:2a
inet 10.1.1.2 netmask 0xffff0000 broadcast 10.1.255.255
inet6 fe80::6a05:caff:fe13:742a%em2 prefixlen 64 scopeid 0x3
inet 10.1.10.3 netmask 0xffffffff broadcast 10.1.10.3
inet 10.1.10.1 netmask 0xffffffff broadcast 10.1.10.1
inet 10.1.10.8 netmask 0xffffffff broadcast 10.1.10.8
inet 10.1.10.10 netmask 0xffffffff broadcast 10.1.10.10
inet 10.1.10.4 netmask 0xffffffff broadcast 10.1.10.4
inet 10.1.10.9 netmask 0xffffffff broadcast 10.1.10.9
inet 10.1.10.7 netmask 0xffffffff broadcast 10.1.10.7
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active


Let me know if any more detail would be helpful or if you have any ideas of
things to check.

Thanks,

Preston
Received on Thu Feb 13 2014 - 17:59:16 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:46 UTC