Re: jails in CURRENT: can not reach hosts on same network

On Wed, Oct 5, 2016 at 5:44 AM, O. Hartmann <ohartman_at_zedat.fu-berlin.de> wrote:
> Hello list.
>
> I struggle with setting up jails on most recent CURRENT.
>
> The machine containing the jails has two NICs (bce0 and bce1). the host itself
> is supposed to own NIC bce0 exclusively - means, the services running on that
> NIC - syslogd, named and others - are bound to that NIC and should not be
> shared with the bce1 or jails bound to bce1.
>
> I followed the instructions given in the most recent version of the handbook
> setting up a jail. So far, so good. The NIC bce1 (the second one) is "aliased"
> with IPs from the local network. forwarding is disabled
> (net.inet.ip.forwarding: 0).
>
> Setup of each jail is straigh forward, with "ip4.addr=" set to the specific IP
> and interface="bce1".
>
> Within a jail, I can not reach an IP on the same network, not even the gateway
> by pinging or doing name resolutions using the DNS server on the local net! The
> curious thing is, by setting "nameserver 8.8.8.8" in /etc/resolv.conf, I can
> ping "outer world systems" and performing name resolutions as well - this
> implies, that the IP pakets are delegated to the local gateway and then further
> to the DNS of Google's. But pinging the local gateway directly (192.168.0.1)
> seems to be prohibited as well as pinging or reching any other IP on the net,
> including the bce0 of the same host (via default gateway?) or any other aliased
> IP.
>
> Since I'm new to jails and the complicated handling with networks, I miss
> something here which is probably not well documented. I found some notes on the
> forum about setfib, FIB, but I lack in the correct manpage to read more about
> this concept, the meaning for a jail and its probable impact in my situation.
>
> Following the suggestion setting
>
> net.add_addr_allfibs=0
>
> in /boot/loader.conf seems to be senseless - after a reboot this OID is always
> set back to 1 (net.add_addr_allfibs=1).
>
> maybe someone has an idea what's wrong in principle with my attempts.
>
> thanks in advance for your patience,
>
> Oliver

Firstly, ping doesn't work in a jail, because jailed processes aren't
allowed to open raw sockets.  To lift that restriction, you can do
"sysctl security.jail.allow_raw_sockets".  Depending on what your
security environment is like, you may or may not want to leave that
set permanently.  You can also control it on a per-jail basis.  If
you're using iocage to manage your jails, just do "iocage set
allow_raw_sockets=1 <jailtag>".  If that doesn't work, then post the
output of "ifconfig".  You shouldn't need to screw with fibs unless
your jails need to use a different gateway than the host.

-Alan