Re: kernel panic by enabling net.inet.ip.random_id

From: Adrian Chadd <adrian.chadd_at_gmail.com>
Date: Wed, 6 Jan 2016 10:15:08 -0800
Why'd you condition the vimage definition? :)



-a


On 6 January 2016 at 06:46, Shawn Webb <shawn.webb_at_hardenedbsd.org> wrote:
> (kgdb) list *(0xffffffff80b5de9e)
> 0xffffffff80b5de9e is in ip_fillid (/usr/src/sys/netinet/ip_id.c:237).
> warning: Source file is more recent than executable.
>
> 232             new_id = 0;
> 233             do {
> 234                     if (new_id != 0)
> 235                             V_random_id_collisions++;
> 236                     arc4rand(&new_id, sizeof(new_id), 0);
> 237             } while (bit_test(V_id_bits, new_id) || new_id == 0);
> 238             bit_clear(V_id_bits, V_id_array[V_array_ptr]);
> 239             bit_set(V_id_bits, new_id);
> 240             V_id_array[V_array_ptr] = new_id;
> 241             V_array_ptr++;
>
> This is the change I made to ip_id.c that caused the underlying kernel
> panic:
> https://github.com/HardenedBSD/hardenedBSD/commit/52d5a93b92097e7a79be8d2e0eb9c1a58b8337d1
>
> Ideally, we should be able to just toggle that variable and all would be
> well. But it seems with the VIMAGE work, something is preventing that.
>
> Thanks,
>
> Shawn
>
> On Tue, Jan 05, 2016 at 06:22:34PM -0800, Adrian Chadd wrote:
>> try list *(0x[address]) .
>>
>> That line is mtx_unlock(), which makes no sense (as mtx_lock succeeded fine.)
>>
>>
>> -a
>>
>>
>> On 5 January 2016 at 18:13, Shawn Webb <shawn.webb_at_hardenedbsd.org> wrote:
>> > Thanks for the quick reply! Here's some more debugging output:
>> >
>> > === Begin Log ===
>> > (kgdb) bt
>> > #0  doadump (textdump=0) at pcpu.h:221
>> > #1  0xffffffff8037c78b in db_dump (dummy=<value optimized out>, dummy2=false, dummy3=0, dummy4=0x0) at /usr/src/sys/ddb/db_command.c:533
>> > #2  0xffffffff8037c57e in db_command (cmd_table=0x0) at /usr/src/sys/ddb/db_command.c:440
>> > #3  0xffffffff8037c314 in db_command_loop () at /usr/src/sys/ddb/db_command.c:493
>> > #4  0xffffffff8037edab in db_trap (type=<value optimized out>, code=0) at /usr/src/sys/ddb/db_main.c:251
>> > #5  0xffffffff80a5c563 in kdb_trap (type=12, code=0, tf=<value optimized out>) at /usr/src/sys/kern/subr_kdb.c:654
>> > #6  0xffffffff80e6b7e1 in trap_fatal (frame=0xfffffe02c33894d0, eva=<value optimized out>) at /usr/src/sys/amd64/amd64/trap.c:829
>> > #7  0xffffffff80e6ba2d in trap_pfault (frame=0xfffffe02c33894d0, usermode=<value optimized out>) at /usr/src/sys/amd64/amd64/trap.c:684
>> > #8  0xffffffff80e6b15f in trap (frame=0xfffffe02c33894d0) at /usr/src/sys/amd64/amd64/trap.c:435
>> > #9  0xffffffff80e4af97 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:234
>> > #10 0xffffffff80b5de9e in ip_fillid (ip=0xfffff8000ef8cb88) at /usr/src/sys/netinet/ip_id.c:237
>> > #11 0xffffffff80b6c41b in ip_output (m=<value optimized out>, opt=<value optimized out>, ro=<value optimized out>, flags=0, imo=0x0, inp=0xfffff8000e66e960) at /usr/src/sys/netinet/ip_output.c:268
>> > #12 0xffffffff80bf0612 in udp_send (so=<value optimized out>, flags=<value optimized out>, m=<value optimized out>, addr=0x0, control=<value optimized out>, td=0xfffff8000ef8cb88) at /usr/src/sys/netinet/udp_usrreq.c:1517
>> > #13 0xffffffff80aa3872 in sosend_dgram (so=0xfffff8000e6422e8, addr=0x0, uio=<value optimized out>, top=0xfffff8000ef8cb00, control=0x0, flags=<value optimized out>, td=0xffffffff81bef2ec) at /usr/src/sys/kern/uipc_socket.c:1164
>> > #13 0xffffffff80aa3872 in sosend_dgram (so=0xfffff8000e6422e8, addr=0x0, uio=<value optimized out>, top=0xfffff8000ef8cb00, control=0x0, flags=<value optimized out>, td=0xffffffff81bef2ec) at /usr/src/sys/kern/uipc_socket.c:1164
>> > #14 0xffffffff80aaa03b in kern_sendit (td=0xfffff8000e4cd9c0, s=6, mp=<value optimized out>, flags=0, control=0x0, segflg=UIO_USERSPACE) at /usr/src/sys/kern/uipc_syscalls.c:906
>> > #15 0xffffffff80aaa336 in sendit (td=0xfffff8000e4cd9c0, s=<value optimized out>, mp=0xfffffe02c3389970, flags=3980) at /usr/src/sys/kern/uipc_syscalls.c:833
>> > #16 0xffffffff80aaa1fd in sys_sendto (td=0x0, uap=<value optimized out>) at /usr/src/sys/kern/uipc_syscalls.c:957
>> > #17 0xffffffff80e6bfdb in amd64_syscall (td=0xfffff8000e4cd9c0, traced=0) at subr_syscall.c:135
>> > #18 0xffffffff80e4b27b in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:394
>> > #19 0x000003e339782e8a in ?? ()
>> > (kgdb) x/i 0xffffffff80b5de9e
>> > 0xffffffff80b5de9e <ip_fillid+142>:     movzbl (%rax,%rcx,1),%esi
>> > (kgdb) info reg
>> > rax            0x0      0
>> > rbx            0x0      0
>> > rcx            0x0      0
>> > rdx            0x0      0
>> > rsi            0x0      0
>> > rdi            0x0      0
>> > rbp            0xfffffe02c3388fe0       0xfffffe02c3388fe0
>> > rsp            0xfffffe02c3388fc8       0xfffffe02c3388fc8
>> > r8             0x0      0
>> > r9             0x0      0
>> > r10            0x0      0
>> > r11            0x0      0
>> > r12            0xffffffff817c0b80       -2122577024
>> > r13            0xffffffff817c1470       -2122574736
>> > r14            0x1      1
>> > r15            0x4      4
>> > rip            0xffffffff80a1fae3       0xffffffff80a1fae3 <doadump+51>
>> > eflags         0x0      0
>> > cs             0x0      0
>> > ss             0x0      0
>> > ds             0x0      0
>> > es             0x0      0
>> > fs             0x0      0
>> > gs             0x0      0
>> > === End Log ===
>> >
>> > Thanks,
>> >
>> > Shawn
>> >
>> > On Tue, Jan 05, 2016 at 06:06:41PM -0800, Adrian Chadd wrote:
>> >> looks like a null pointer deference. What's kgdb show at that IP?
>> >>
>> >>
>> >> -a
>> >>
>> >>
>> >> On 5 January 2016 at 17:57, Shawn Webb <shawn.webb_at_hardenedbsd.org> wrote:
>> >> > Hey All,
>> >> >
>> >> > Here's a kernel panic I'm experiencing by enabling net.inet.ip.random_id
>> >> > at boot.
>> >> >
>> >> > I'm on latest HEAD on amd64 in bhyve. I'll soon-ish be testing on native
>> >> > hardware with VIMAGE enabled.
>> >> >
>> >> > === Begin Log ===
>> >> > Kernel page fault with the following non-sleepable locks held:
>> >> > exclusive sleep mutex ip_id_mtx (ip_id_mtx) r = 0 (0xffffffff81c54830) locked _at_ /usr/src/sys/netinet/ip_id.c:227
>> >> > stack backtrace:
>> >> > #0 0xffffffff80a79620 at witness_debugger+0x70
>> >> > #1 0xffffffff80a7a937 at witness_warn+0x3d7
>> >> > #2 0xffffffff80e6b887 at trap_pfault+0x57
>> >> > #3 0xffffffff80e6b15f at trap+0x4bf
>> >> > #4 0xffffffff80e4af97 at calltrap+0x8
>> >> > #5 0xffffffff80b6c41b at ip_output+0x16b
>> >> > #6 0xffffffff80b68e82 at icmp_reflect+0x5b2
>> >> > #7 0xffffffff80b6883f at icmp_error+0x46f
>> >> > #8 0xffffffff80beeb12 at udp_input+0x982
>> >> > #9 0xffffffff80b69d1d at ip_input+0x17d
>> >> > #10 0xffffffff80b08ba1 at netisr_dispatch_src+0x81
>> >> > #11 0xffffffff80afecce at ether_demux+0x15e
>> >> > #12 0xffffffff80affa14 at ether_nh_input+0x344
>> >> > #13 0xffffffff80b08ba1 at netisr_dispatch_src+0x81
>> >> > #14 0xffffffff80afefcf at ether_input+0x4f
>> >> > #15 0xffffffff8089a5c3 at vtnet_rxq_eof+0x823
>> >> > #16 0xffffffff8089b2ce at vtnet_rx_vq_intr+0x4e
>> >> > #17 0xffffffff809e9ba6 at intr_event_execute_handlers+0x96
>> >> >
>> >> >
>> >> > Fatal trap 12: page fault while in kernel mode
>> >> > cpuid = 6; apic id = 06
>> >> > fault virtual address   = 0x5bd
>> >> > fault code              = supervisor read data, page not present
>> >> > instruction pointer     = 0x20:0xffffffff80b5de9e
>> >> > stack pointer           = 0x28:0xfffffe02b8d483e0
>> >> > frame pointer           = 0x28:0xfffffe02b8d48410
>> >> > code segment            = base 0x0, limit 0xfffff, type 0x1b
>> >> >                         = DPL 0, pres 1, long 1, def32 0, gran 1
>> >> > processor eflags        = interrupt enabled, resume, IOPL = 0
>> >> > current process         = 12 (irq265: virtio_pci0)
>> >> > [ thread pid 12 tid 100040 ]
>> >> > Stopped at      ip_fillid+0x8e: movzbl  (%rax,%rcx,1),%esi
>> >> > === End Log ===
>> >> >
>> >> > Thanks,
>> >> >
>> >> > --
>> >> > Shawn Webb
>> >> > HardenedBSD
>> >> >
>> >> > GPG Key ID:          0x6A84658F52456EEE
>> >> > GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89  3D9E 6A84 658F 5245 6EEE
>> >
>> > --
>> > Shawn Webb
>> > HardenedBSD
>> >
>> > GPG Key ID:          0x6A84658F52456EEE
>> > GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89  3D9E 6A84 658F 5245 6EEE
>
> --
> Shawn Webb
> HardenedBSD
>
> GPG Key ID:          0x6A84658F52456EEE
> GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89  3D9E 6A84 658F 5245 6EEE
Received on Wed Jan 06 2016 - 17:15:09 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:02 UTC