Re: FreeBSD CI Weekly Report 2019-06-09

On 15 Jun 2019, at 11:35, Kristof Provost wrote:
> On 12 Jun 2019, at 16:49, Li-Wen Hsu wrote:
>> * https://ci.freebsd.org/job/FreeBSD-head-i386-test/
>>     * Same as amd64:
>>         * sys.netinet.socket_afinet.socket_afinet_bind_zero
>>     * Others:
>>         * sys.netpfil.pf.forward.v6
>>         * sys.netpfil.pf.forward.v4
>>         * sys.netpfil.pf.set_tos.v4
>
> I’ve finally gotten around to taking a look at this, and it appears 
> to not be a pf problem. forward:v4 already fails at its sanity check, 
> before it configures pf.
>
> It creates a vnet jail, telling it to route traffic through, and then 
> we run a sanity check with pft_ping.py.
> Scapy tries to resolve the MAC address of the gateway (jail, 
> 192.0.2.1). The jail replies, but scapy never picks up the reply, so 
> the traffic looks like this:
>
> 	13:19:29.953468 02:be:b4:57:9f:0a > ff:ff:ff:ff:ff:ff, ethertype ARP 
> (0x0806), length 42: Request who-has 192.0.2.2 tell 192.0.2.1, length 
> 28
> 	13:19:29.953572 02:be:b4:57:9f:0b > 00:a0:98:b2:48:59, ethertype ARP 
> (0x0806), length 42: Reply 192.0.2.2 is-at 02:be:b4:57:9f:0b, length 
> 28
> 	13:19:32.082843 02:be:b4:57:9f:0a > ff:ff:ff:ff:ff:ff, ethertype IPv4 
> (0x0800), length 52: 192.0.2.1 > 198.51.100.3: ICMP echo request, id 
> 0, seq 0, length 18
>
> The jail doesn’t forward the broadcast ICMP echo request and the 
> test fails.
>
> My current guess is that it’s related to bpf. It’s interesting to 
> note that it fails on i386, but succeeds on amd64.
>
I’ve done a little dtracing, and I think that points at bpf too:

	#!/usr/sbin/dtrace -s

	fbt:kernel:bpf_buffer_uiomove:entry
	{
	tracemem(arg1, 1500, arg2);
	stack();
	}

Results in:

	  1  49539         bpf_buffer_uiomove:entry
	             0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f  
0123456789abcdef
	         0: ce 0e 05 5d 17 ea 00 00 2a 00 00 00 2a 00 00 00  
...]....*...*...
	        10: 12 00 ff ff ff ff ff ff 02 fd 10 30 e6 0a 08 06  
...........0....
	        20: 00 01 08 00 06 04 00 01 00 a0 98 b2 48 59 c0 00  
............HY..
	        30: 02 01 00 00 00 00 00 00 c0 00 02 02 ce 0e 05 5d  
...............]
	        40: 60 ea 00 00 2a 00 00 00 2a 00 00 00 12 00 00 a0  
`...*...*.......
	        50: 98 b2 48 59 02 fd 10 30 e6 0b 08 06 00 01 08 00  
..HY...0........
	        60: 06 04 00 02 02 fd 10 30 e6 0b c0 00 02 02 00 a0  
.......0........
	        70: 98 b2 48 59 c0 00 02 01                          ..HY....

	              kernel`bpfread+0x137
	              kernel`dofileread+0x6d
	              kernel`kern_readv+0x3b
	              kernel`sys_read+0x48
	              kernel`syscall+0x2b4
	              0xffc033b7

So, we see the ARP request through bpf, but we don’t see the reply, 
despite tcpdump capturing it. I have no idea how that’d happen, so 
I’d very much like someone more familiar with bpf to take a look at 
this problem.

Regards,
Kristof