Re: Cam panic on r271170

On 9/17/2014 10:39 AM, Bryan Drewery wrote:
> On 9/16/2014 9:28 PM, Bryan Drewery wrote:
>> I've been getting this quite frequently on head recently. I have dumps
>> if anyone is interested in more information.
>>
>>> Fatal trap 9: general protection fault while in kernel mode
>>> cpuid = 10; Memory modified after free 0xfffff8003e0b0800(2040)
>>> val=ffffffff _at_ 0xfffff8003e0b0808
>>> apanic: Most recently used by CAM CCB
>>>
>>> cpuid = 6
>>> KDB: stack backtrace:
>>> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
>>> 0xfffffe124735b4c0
>>> kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe124735b570
>>> vpanic() at vpanic+0x189/frame 0xfffffe124735b5f0
>>> panic() at panic+0x43/frame 0xfffffe124735b650
>>> mtrash_ctor() at mtrash_ctor+0x8a/frame 0xfffffe124735b680
>>> uma_zalloc_arg() at uma_zalloc_arg+0x4f1/frame 0xfffffe124735b6f0
>>> malloc() at malloc+0x192/frame 0xfffffe124735b740
>>> xpt_run_allocq() at xpt_run_allocq+0xb5/frame 0xfffffe124735b780
>>> adastrategy() at adastrategy+0x117/frame 0xfffffe124735b7b0
>>> g_io_request() at g_io_request+0x3b7/frame 0xfffffe124735b810
>>> g_part_start() at g_part_start+0x2b7/frame 0xfffffe124735b890
>>> g_io_request() at g_io_request+0x3b7/frame 0xfffffe124735b8f0
>>> g_io_request() at g_io_request+0x3b7/frame 0xfffffe124735b950
>>> vdev_geom_io_start() at vdev_geom_io_start+0x137/frame 0xfffffe124735b970
>>> zio_vdev_io_start() at zio_vdev_io_start+0x49f/frame 0xfffffe124735b9d0
>>> zio_execute() at zio_execute+0x204/frame 0xfffffe124735ba30
>>> vdev_queue_io_done() at vdev_queue_io_done+0x180/frame 0xfffffe124735ba80
>>> zio_vdev_io_done() at zio_vdev_io_done+0x11d/frame 0xfffffe124735bac0
>>> zio_execute() at zio_execute+0x204/frame 0xfffffe124735bb20
>>> taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame
>>> 0xfffffe124735bb80
>>> taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame
>>> 0xfffffe124735bbb0
>>> fork_exit() at fork_exit+0x84/frame 0xfffffe124735bbf0
>>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe124735bbf0
>>> --- trap 0, rip = 0, rsp = 0xfffffe124735bcb0, rbp = 0 ---
>>> KDB: enter: panic
>>> [ thread pid 0 tid 100571 ]
>>> Stopped at      kdb_enter+0x3e: movq    $0,kdb_why
>>
>>
> 
> I also had this one recently:
> 
>> #8  0xffffffff80d1a162 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:231
>> #9  0xffffffff802e52c4 in xpt_path_periph (path=0xdeadc0dedeadc0de) at /usr/src/sys/cam/cam_xpt.c:3738
>> #10 0xffffffff802dfe62 in cam_periph_error (ccb=0xfffff8003e0b6000, camflags=CAM_FLAG_NONE, sense_flags=0, save_ccb=0x0) at /usr/src/sys/cam/cam_periph.c:1602
>> #11 0xffffffff803057e4 in adadone (periph=0xfffff8003e09b700, done_ccb=0xfffff8003e0b6000) at /usr/src/sys/cam/ata/ata_da.c:1877
>> #12 0xffffffff802e6e44 in xpt_done_process (ccb_h=0xfffff8003e0b6000) at /usr/src/sys/cam/cam_xpt.c:5245
>> #13 0xffffffff80394d59 in ahci_ch_intr_direct (arg=<value optimized out>) at /usr/src/sys/dev/ahci/ahci.c:1132
>> #14 0xffffffff80390ff1 in ahci_intr (data=<value optimized out>) at /usr/src/sys/dev/ahci/ahci.c:417
>> #15 0xffffffff808ea5d3 in intr_event_execute_handlers (p=<value optimized out>, ie=0xfffff8000f725d00) at /usr/src/sys/kern/kern_intr.c:1252
>> #16 0xffffffff808eafb6 in ithread_loop (arg=0xfffff8000f6dea60) at /usr/src/sys/kern/kern_intr.c:1265
>> #17 0xffffffff808e7fc4 in fork_exit (callout=0xffffffff808eaf10 <ithread_loop>, arg=0xfffff8000f6dea60, frame=0xfffffe1245083c00) at /usr/src/sys/kern/kern_fork.c:977
>> #18 0xffffffff80d1a69e in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:605
> 
> 

Another, with much more information here:
https://people.freebsd.org/~bdrewery/cam.panic.txt

This was with memguard (vm.memguard.desc="CAM CCB") and a KASSERT in
malloc(9) and uma_zalloc_arg() to prevent M_WAITOK in non-sleepable
threads. Neither triggered.

It seems to be when syncing to the SSD ZFS log I have.

-- 
Regards,
Bryan Drewery