Re: VIMAGE: vnet, epair and lots of jails on bridgeX - routing

From: O. Hartmann <ohartmann_at_walstatt.org>
Date: Mon, 12 Feb 2018 09:29:01 +0100
On Sat, 10 Feb 2018 11:52:18 +0100
Olivier Cochard-Labbé <olivier_at_freebsd.org> wrote:

> On Sat, Feb 10, 2018 at 8:52 AM, O. Hartmann <ohartmann_at_walstatt.org> wrote:
> 
> >
> > The moment any of the bridges gets an additional member epair interface
> > (so the bridge
> > has at least three members including the on reaching into the virtual
> > router jail) the
> > vbridge seems to operate unpredictable (to me). Pinging jails memeber of
> > that vbridge
> > are unreachable.
> >
> >  
> ​First idea:
> Did you try with a more simple setup, like with just 3 jails members of one
> bridge ?
> => I've tried it on a -head, and all 4 members  (3 jails and the host)  
> reach to communicate.

Yes, I did.

I used to setup one bridge (bridge0) and three jails. Each jail owns its
epairXXa device with IP assigned. epairXXb of each jail is member of the
bridge. Bridge is up, epairXXb is up.

> 
> Second idea:
> Can you check that all epairs have different MAC address each ?​
> I hit this bug: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=176671

Very good shot! Thanks. No, they do not have all of them unique MAC adrresses
and in some occassions members of the very same bridge have the very same MAC
addresses, mostly the a-part of the epair:

From console:
[...]
ng_ether_ifnet_arrival_event: can't re-name node epair10250a
==>> epair20128a: Ethernet address: 02:ce:d0:00:07:0a
epair20128b: Ethernet address: 02:ce:d0:00:13:0b
epair20128a: link state changed to UP
epair20128b: link state changed to UP
epair20128b: promiscuous mode enabled
ng_ether_ifnet_arrival_event: can't re-name node epair20128a
==>> epair20129a: Ethernet address: 02:ce:d0:00:07:0a
epair20129b: Ethernet address: 02:ce:d0:00:14:0b
epair20129a: link state changed to UP
epair20129b: link state changed to UP
epair20129b: promiscuous mode enabled
ng_ether_ifnet_arrival_event: can't re-name node epair20129a

epair20XXX are member of bridge2 and dedicated epairs of jails.

The following is the desastrous picture of bridge1:

[...]
==>> epair235a: Ethernet address: 02:ce:d0:00:07:0a
epair235b: Ethernet address: 02:ce:d0:00:0d:0b
epair235a: link state changed to UP
epair235b: link state changed to UP
epair235b: promiscuous mode enabled
ng_ether_ifnet_arrival_event: can't re-name node epair235a
==>> epair237a: Ethernet address: 02:ce:d0:00:07:0a
epair237b: Ethernet address: 02:ce:d0:00:0e:0b
epair237a: link state changed to UP
epair237b: link state changed to UP
epair237b: promiscuous mode enabled
ng_ether_ifnet_arrival_event: can't re-name node epair237a
==>> epair251a: Ethernet address: 02:ce:d0:00:07:0a
epair251b: Ethernet address: 02:ce:d0:00:0f:0b
epair251a: link state changed to UP
epair251b: link state changed to UP
epair251b: promiscuous mode enabled
ng_ether_ifnet_arrival_event: can't re-name node epair251a
==>> epair238a: Ethernet address: 02:ce:d0:00:07:0a
epair238b: Ethernet address: 02:ce:d0:00:10:0b
epair238a: link state changed to UP
epair238b: link state changed to UP
epair238b: promiscuous mode enabled
ng_ether_ifnet_arrival_event: can't re-name node epair238a
[...]


This is on CURRENT (FreeBSD 12.0-CURRENT #36 r329150: Mon Feb 12 06:30:47 CET
2018 amd64).

I did check on Friday in the bureau and didn't catch it since I was checking on
each jail, but the console log accessed via dmesg revealed the problem very
easily.

I did a check on the 11.1-RELENG-p6 box and I got the same picture there,
different, but very similar setup. 

So I didn't see it in the masses of epairs our setup requires :-(

I'll go now for setting MAC addresses manually and check functionality again.

> 
> Regards,
> 
> Olivier

Kind regards,

Oliver
Received on Mon Feb 12 2018 - 07:29:11 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:14 UTC