Re: NULL pointer crash in exit1() when running certain Linux binaries

On Sun, Jun 23, 2013 at 12:24:55AM +0200, Hans Petter Selasky wrote:
> Hi,
> 
> The following crash has been observed using FreeBSD 9-stable amd64:
> 
> instruction pointer     = 0x20:0xffffffff81765bb6
[..]
> KDB: stack backtrace:
> #0 0xffffffff809553b6 at kdb_backtrace+0x66
> #1 0xffffffff8091c72e at panic+0x1ce
> #2 0xffffffff80cabb40 at trap_fatal+0x290
> #3 0xffffffff80cabea1 at trap_pfault+0x211
> #4 0xffffffff80cac454 at trap+0x344
> #5 0xffffffff80c957e3 at calltrap+0x8
> #6 0xffffffff808e68ab at exit1+0x1bb
> #7 0xffffffff81773dcf at linux_exit_group+0xaf
> #8 0xffffffff80d2728e at ia32_syscall+0x57e
> #9 0xffffffff80c95db1 at Xint0x80_syscall+0x91
> Uptime: 40m36s
> 
> #7  0xffffffff81765bb6 in linux_proc_exit (arg=<value optimized out>,
>     p=<value optimized out>)
>     at /usr/img/freebsd.9/sys/modules/linux/../../compat/linux/linux_emul.c:326
> #8  0xffffffff808e68ab in exit1 (td=0xfffffe0130cce490,
>     rv=<value optimized out>) at
> /usr/img/freebsd.9/sys/kern/kern_exit.c:261
> #9  0xffffffff81773dcf in linux_exit_group (td=0xfffffe0130cce490,
>     args=0xffffff81225cbb70)
>     at /usr/img/freebsd.9/sys/modules/linux/../../compat/linux/linux_misc.c:1686
> #10 0xffffffff80d2728e in ia32_syscall (frame=0xffffff81225cbc00)
>     at subr_syscall.c:135
> #11 0xffffffff80c95db1 in Xint0x80_syscall () at ia32_exception.S:73
> #12 0x00000000080f2047 in ?? ()
> Previous frame inner to this frame (corrupt stack?)
> 
> 
>         /* Are we a task leader? */
>         if (p == p->p_leader) {
>      364:       4d 8b a6 18 04 00 00    mov    0x418(%r14),%r12
>      36b:       4d 39 f4                cmp    %r14,%r12
>      36e:       0f 84 c2 0d 00 00       je     1136 <exit1+0xf36>
>         /*
>          * Check if any loadable modules need anything done at
> process exit.
>          * E.g. SYSV IPC stuff
>          * XXX what if one of these generates an error?
>          */
>         EVENTHANDLER_INVOKE(process_exit, p);
>      374:       48 c7 c7 00 00 00 00    mov    $0x0,%rdi
>                         377: R_X86_64_32S       .rodata.str1.1+0xf
>      37b:       e8 00 00 00 00          callq  380 <exit1+0x180>
>                         37c: R_X86_64_PC32
> eventhandler_find_list+0xfffffffffffffffc
>      380:       48 85 c0                test   %rax,%rax
>      383:       49 89 c4                mov    %rax,%r12
>      386:       0f 84 e3 00 00 00       je     46f <exit1+0x26f>
>      38c:       8b 40 0c                mov    0xc(%rax),%eax
>      38f:       4d 8b 6c 24 40          mov    0x40(%r12),%r13
>      394:       83 c0 01                add    $0x1,%eax
>      397:       4d 85 ed                test   %r13,%r13
>      39a:       41 89 44 24 0c          mov    %eax,0xc(%r12)
>      39f:       0f 84 97 00 00 00       je     43c <exit1+0x23c>
>      3a5:       4d 8d 7c 24 10          lea    0x10(%r12),%r15
>      3aa:       eb 40                   jmp    3ec <exit1+0x1ec>
>      3ac:       0f 1f 40 00             nopl   0x0(%rax)
>      3b0:       4c 89 f6                mov    %r14,%rsi
>      3b3:       49 8b 7d 18             mov    0x18(%r13),%rdi
>      3b7:       41 ff 55 20             callq  *0x20(%r13)
> ^^^ NULL pointer
>      3bb:       65 48 8b 34 25 00 00    mov    %gs:0x0,%rsi
> 
>      3c2:       00 00
>      3c4:       48 89 d8                mov    %rbx,%rax
>      3c7:       f0 49 0f b1 74 24 28    lock cmpxchg %rsi,0x28(%r12)
>      3ce:       0f 94 c0                sete   %al
>      3d1:       84 c0                   test   %al,%al
>      3d3:       74 4a                   je     41f <exit1+0x21f>
>      3d5:       8b 3d 00 00 00 00       mov    0x0(%rip),%edi
> # 3db <exit1+0x1db>
> 
> 

I'm not sure I follow what you posted here. If the creash was indeed in
linux_proc_exit while dereferncing em pointer, I think the issue is
known:
http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/142082

(btw, at the time of reporting it it was actually kernel memory
disclosure :->)

I have a quick hack to get around it:
http://people.freebsd.org/~mjg/patches/linux-emuldata-race-hack.diff

I believe dchagin_at_ did a lot of reworking in this area in his linux
branch, so I guess this hack will be sufficent for now.

-- 
Mateusz Guzik <mjguzik gmail.com>