Re: VIMAGE: vnet, epair and lots of jails on bridgeX - routing

On Sat, 10 Feb 2018 08:52:21 +0100
"O. Hartmann" <ohartmann_at_walstatt.org> wrote:

> Am Fri, 09 Feb 2018 16:43:17 +0000
> "Bjoern A. Zeeb" <bzeeb-lists_at_lists.zabbadoz.net> schrieb:
> 
> > On 9 Feb 2018, at 16:22, O. Hartmann wrote:
> >   
> > > Am Thu, 8 Feb 2018 09:31:15 +0100
> > > "O. Hartmann" <ohartmann_at_walstatt.org> schrieb:
> > >
> > > Is this problem to trivial?    
> > 
> > I read through it yesterday and found myself in the position that I
> > need a whiteboard or paper and pencil or an ASCII art of your
> > situation.  But by the time I made it to the question I was
> > basically lost.  Could you massively simplify this and maybe
> > produce the ASCII art?
> > 
> > /bz
> > _______________________________________________
> > freebsd-current_at_freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-current
> > To unsubscribe, send any mail to
> > "freebsd-current-unsubscribe_at_freebsd.org"  
> 
> All right.
> 
> I'm not much of an artist and at this very moment, I haven't much
> experience with neat ASCII art tools. But I'll provide a sketch
> later, but I also will simplify  the situation.
> 
> Consider three "vswitches", basically based on the creation of
> bridges, bridge0, bridge1, bridge2. Create at least three individual
> vnet-jails attached to each vbridge. Those jails have epair pseudo
> devices. The jail itself owns the "a-part" of the epair and the
> b-part is "member of the bridge". Each jail's epairXXXa has an IP
> assigned of the network the vswitch is part of. I mention a- and
> b-part of the epair here, because I thought it could matter, but I
> think for symmetry reasons it doesn't.
> 
> Now consider a further, special jail. This jail is supposed to have
> three epair devices, each one is reaching into one of the vbridges.
> This jail is the router/routing jail. Later, this jail should filter
> via IPFW the traffic between the three vbridges according to rules,
> but this doesn't matter here, beacuase the basics are not working as
> expected.
> 
> Now the problems. It doesn't matter on which jail of the three
> vswitches I login, the moment a vbridge has more than two member
> epairs (one  is alway member of the routing jail, now consider a
> database jail and a webserver jail), pinging each jail or the routing
> jail fails. It works sometimes for a couple of ICMP packets and then
> stops.
> 
> If each vbridge has only one member jail, I have NO PROBLEMS
> traversing accordingly to the static routing rules from one vbridge
> to any other, say from vbridge1 to vbridge0 or vbridge2 and any
> permutation of that.
> 
> The moment any of the bridges gets an additional member epair
> interface (so the bridge has at least three members including the on
> reaching into the virtual router jail) the vbridge seems to operate
> unpredictable (to me). Pinging jails memeber of that vbridge are
> unreachable.
> 
> Technical information:
> 
> The kernel has options IPFIREWALL, VIMAGE. The host's ipfw (kernel)
> declines packets by default. Each jail is configured to have ipfw
> "open".
> 
> Thanks for the patience.

If you could provide a script which sets up the topology you described
in two lengthy posts then others could reproduce this, and your chances
of getting useful feedback would certainly increase.

We also have a graphical tool (https://github.com/imunes/imunes) which
can set up a topology like you described in a few clicks of a mouse,
albeit using netgraph and ng_eiface instead of epairs, but I assume this
is irellevant as long as you are not aiming for maximum packet
throughputs.  If you attempt to use this tool, note that selecting
"stpswitch" will create if_bridge instances, whereas "lanswitch"
creates ng_bridge instances.

Good luck,

Marko


> 
> Kind regards,
> 
> O. Hartmann