Re: ifnet use after free

On 25 Aug 2018, at 0:47, Kristof Provost wrote:

> On 25 Aug 2018, at 0:26, Matthew Macy wrote:
>> On Fri, Aug 24, 2018 at 15:25 Shawn Webb <shawn.webb_at_hardenedbsd.org> 
>> wrote:
>>
>>> Hey All,
>>>
>>> Somewhere in the last month or so, a use after free was introduced. 
>>> I
>>> don't have the time right now to bisect the commits and figure out
>>> which commit introduced the breakage. Attached is the core.txt 
>>> (which
>>> seems nonsensical because the dump is reporting on a different
>>> thread). If the core.txt gets scrubbed, I've posted it here:
>>> https://gist.github.com/796ea88cec19a1fd2a85f4913482286a
>>>
>>
>> Do you have any guidance on how to reproduce? The hardenedbsd rev 
>> isn’t
>> useful - the svn commit that it’s based against is what is needed.
>>
> For what it’s worth, it’s not a hardenedbsd thing. I’ve been 
> chasing the same one (same offset, same allocation size, same most 
> recent user). Something gets set to zero/NULL. 8 bytes on amd64, so 
> presumably a pointer.
>
> I currently only trigger it on a development branch, but I’ll see if 
> I can clean that up into something I can share tomorrow.
>
> In my test scenario it happens after shutdown of a vnet jail with a 
> few interfaces in it (including a pfsync interface which will 
> disappear with the jail), and new jails are started. It’s pretty 
> reliable.
>
> At a guess something’s wrong with the delayed cleanup of ifnets and 
> vnet shutdown.
>
I see this:

	Memory modified after free 0xfffff800623ab000(2040) val=0 _at_ 
0xfffff800623ab398
	panic: Most recently used by ifnet

	cpuid = 7
	time = 1535199812
	KDB: stack backtrace:
	db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 
0xfffffe008c8e13c0
	vpanic() at vpanic+0x1a3/frame 0xfffffe008c8e1420
	panic() at panic+0x43/frame 0xfffffe008c8e1480
	mtrash_ctor() at mtrash_ctor+0x81/frame 0xfffffe008c8e14a0
	uma_zalloc_arg() at uma_zalloc_arg+0x72c/frame 0xfffffe008c8e1510
	malloc() at malloc+0x9a/frame 0xfffffe008c8e1560
	if_alloc() at if_alloc+0x23/frame 0xfffffe008c8e1590
	epair_clone_create() at epair_clone_create+0x239/frame 
0xfffffe008c8e1610
	if_clone_createif() at if_clone_createif+0x4a/frame 0xfffffe008c8e1660
	ifioctl() at ifioctl+0x852/frame 0xfffffe008c8e1750
	kern_ioctl() at kern_ioctl+0x2ba/frame 0xfffffe008c8e17b0
	sys_ioctl() at sys_ioctl+0x15e/frame 0xfffffe008c8e1880
	amd64_syscall() at amd64_syscall+0x28c/frame 0xfffffe008c8e19b0
	fast_syscall_common() at fast_syscall_common+0x101/frame 
0xfffffe008c8e19b0
	--- syscall (54, FreeBSD ELF64, sys_ioctl), rip = 0x80047b74a, rsp = 
0x7fffffffe208, rbp = 0x7fffffffe250 ---
	KDB: enter: panic
	[ thread pid 1426 tid 100466 ]
	Stopped at      kdb_enter+0x3b: movq    $0,kdb_why
	db>

It does require a couple of bug fixes in pfsync to trigger. You can get 
them from the pfsync_vnet branch in 
https://github.com/kprovost/freebsd/tree/pfsync_vnet

After that:
kldload pfsync
pkg install scapy
cd /usr/tests/sys/netpfil/pf
kyua test

It should panic reliably.

Regards,
Kristof