I ran into severe problems on CURRENT ( FreeBSD 13.0-CURRENT #193 r343521: Mon Jan 28 10:26:36 CET 2019 amd64), VIMAGE enabled host with jails utilizing IPv6. Scenario: The main host has two Braodcom (bce0|1) NICs. bce0 is the physical NIC attached to a routed/switched network for the main host. bce1 is also attached to the same network, but via another port on the switch (Cisco). Gatewaying is not allowed on the main host. bce1 is also member of bridge0. The main host hosts a bunch of vnet/VIMAGE jails (~12): each jail has its "epair" pseudo NIC, of which the a-part (epairXXa) is owned by the jail and the b-part is member of the bridge0. NIC bce1 ensures the connection to the physical network. On all hosts IPV6 is enabled. All host use an ULA IPV6 address. All hosts and jails use FreeBSD's native IPFW as their IP filter. bridge0 is configured to not filter on Level 2 (ethernet). IPFW is configured on each jail via rc.conf and script "WORKSTATION". For example, services are allowed by the rc.conf-line: (main host) firewall_type="WORKSTATION" firewall_myservices="22/tcp 53/udp 80/tcp 443/tcp ..." firewall_allowservices="192.168.255.0/24 fdff:dead:beef::/48" firewall_trusted="192.168.255.2 fdff:dead:beef::34 ..." and similar for the jails. Problem: I can not ping (icmp IPv4) any jail from the main host, any host on the regular internet (i.e. google.de/google.com and so on) or any jail, nor can I ping from inside a jail any host or other jail. Since we use some ICINGA2 facilities, pinging is essential. The weird part: ping6 is working perfectly! Alos, any non-ICMPv4 connection is performed well (ssh, http-80, http-443, NFS via 2049 and so on). Disabling IPFW or switch to "OPEN" on a jail or the main host makes things work again. A very similar setup on a host without jails, using also rc.conf for configuring the IPFW paketfilter doesn't reveal such a misbehaviour. The ruleset on a JAIL with ipfw script "WORKSTATION" configured, which is NOT working (icmp doesn't work as expected), looks like this: [...] service ipfw restart Flushed all rules. 00100 allow ip from any to any via lo0 00200 deny ip from any to 127.0.0.0/8 00300 deny ip from 127.0.0.0/8 to any 00400 deny ip from any to ::1 00500 deny ip from ::1 to any 00600 allow ipv6-icmp from :: to ff02::/16 00700 allow ipv6-icmp from fe80::/10 to fe80::/10 00800 allow ipv6-icmp from fe80::/10 to ff02::/16 00900 allow ipv6-icmp from any to any icmp6types 1 01000 allow ipv6-icmp from any to any icmp6types 2,135,136 00000 check-state :default 01200 allow tcp from me to any established 00000 allow tcp from me to any setup keep-state :default 00000 allow udp from me to any keep-state :default 00000 allow icmp from me to any keep-state :default 00000 allow ipv6-icmp from me to any keep-state :default 01700 allow udp from 0.0.0.0 68 to 255.255.255.255 67 out 01800 allow udp from any 67 to me 68 in 01900 allow udp from any 67 to 255.255.255.255 68 in 02000 allow udp from fe80::/10 to me 546 in 02100 allow icmp from any to any icmptypes 8 02200 allow ipv6-icmp from any to any icmp6types 128,129 02300 allow icmp from any to any icmptypes 3,4,11 02400 allow ipv6-icmp from any to any icmp6types 3 02500 allow tcp from 192.168.255.0/24 to me 22 02600 allow tcp from 192.168.255.0/24 to me 80 02700 allow tcp from 192.168.255.0/24 to me 443 02800 allow tcp from fdff:dead:beef::/48 to me 22 02900 allow tcp from fdff:dead:beef::/48 to me 80 03000 allow tcp from fdff:dead:beef::/48 to me 443 65000 count ip from any to any 65100 deny { tcp or udp } from any to any 135-139,445 in 65200 deny { tcp or udp } from any to any 1026,1027 in 65300 deny { tcp or udp } from any to any 1433,1434 in 65400 deny ip from any to 255.255.255.255 65500 deny ip from any to 224.0.0.0/24 in 65500 deny udp from any to any 520 in 65500 deny tcp from any 80,443 to any 1024-65535 in 65500 deny ip from any to any Firewall rules loaded. [...] I can not see the problem here in the configuration :-( On the main host (owner of bce1 and bridge0), net.link.bridge looks like: # sysctl net.link.bridge net.link.bridge.ipfw: 0 net.link.bridge.allow_llz_overlap: 0 net.link.bridge.inherit_mac: 1 net.link.bridge.log_stp: 1 net.link.bridge.pfil_local_phys: 0 net.link.bridge.pfil_member: 0 net.link.bridge.ipfw_arp: 0 net.link.bridge.pfil_bridge: 0 net.link.bridge.pfil_onlyip: 0 Stopping all jails, destroying all epairs and bridge0 doesn't change anything. The problems occured when IPv6 came into play on the specific host in question. Does anyone have any ideas? I'm out of ideas. Thanks in advance, OliverReceived on Mon Jan 28 2019 - 11:44:15 UTC
This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:20 UTC