Re: VIMAGE: vnet, epair and lots of jails on bridgeX - routing

Am Fri, 09 Feb 2018 16:43:17 +0000
"Bjoern A. Zeeb" <bzeeb-lists_at_lists.zabbadoz.net> schrieb:

> On 9 Feb 2018, at 16:22, O. Hartmann wrote:
> 
> > Am Thu, 8 Feb 2018 09:31:15 +0100
> > "O. Hartmann" <ohartmann_at_walstatt.org> schrieb:
> >
> > Is this problem to trivial?  
> 
> I read through it yesterday and found myself in the position that I need 
> a whiteboard or paper and pencil or an ASCII art of your situation.  But 
> by the time I made it to the question I was basically lost.  Could you 
> massively simplify this and maybe produce the ASCII art?
> 
> /bz
> _______________________________________________
> freebsd-current_at_freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"

All right.

I'm not much of an artist and at this very moment, I haven't much experience with neat
ASCII art tools. But I'll provide a sketch later, but I also will simplify  the situation.

Consider three "vswitches", basically based on the creation of bridges, bridge0, bridge1,
bridge2. Create at least three individual vnet-jails attached to each vbridge. Those
jails have epair pseudo devices. The jail itself owns the "a-part" of the epair and the
b-part is "member of the bridge". Each jail's epairXXXa has an IP assigned of the network
the vswitch is part of. I mention a- and b-part of the epair here, because I thought it
could matter, but I think for symmetry reasons it doesn't.

Now consider a further, special jail. This jail is supposed to have three epair devices,
each one is reaching into one of the vbridges. This jail is the router/routing jail.
Later, this jail should filter via IPFW the traffic between the three vbridges according
to rules, but this doesn't matter here, beacuase the basics are not working as expected.

Now the problems. It doesn't matter on which jail of the three vswitches I login, the
moment a vbridge has more than two member epairs (one  is alway member of the routing
jail, now consider a database jail and a webserver jail), pinging each jail or the
routing jail fails. It works sometimes for a couple of ICMP packets and then stops.

If each vbridge has only one member jail, I have NO PROBLEMS traversing accordingly to
the static routing rules from one vbridge to any other, say from vbridge1 to vbridge0 or
vbridge2 and any permutation of that.

The moment any of the bridges gets an additional member epair interface (so the bridge
has at least three members including the on reaching into the virtual router jail) the
vbridge seems to operate unpredictable (to me). Pinging jails memeber of that vbridge
are unreachable.

Technical information:

The kernel has options IPFIREWALL, VIMAGE. The host's ipfw (kernel) declines packets by
default. Each jail is configured to have ipfw "open".

Thanks for the patience.

Kind regards,

O. Hartmann