Re: 7.0-BETA4 - witness_warn on boot

From: Scott Long <scottl_at_samsco.org>
Date: Tue, 8 Jan 2008 23:34:28 -0700
Thanks for the reminder, I'll take care of it.

Scott

On Jan 7, 2008, at 8:01 AM, Michael Jung wrote:

>
>
> -----Original Message-----
> From: Scott Long [mailto:scottl_at_samsco.org]
> Sent: Sunday, December 16, 2007 2:48 AM
> To: Michael Jung
> Cc: freebsd-current_at_freebsd.org
> Subject: Re: 7.0-BETA4 - witness_warn on boot
>
> This simple patch should fix it.  This code path is pretty rare,  
> though,
> I'm frankly shocked that you're running into it.
>
> --- cam_xpt.c   20 Oct 2007 23:23:12 -0000      1.191
> +++ cam_xpt.c   16 Dec 2007 07:46:22 -0000
> _at__at_ -3862,6 +3862,7 _at__at_
>                                                    &work_ccb->ccb_h,
>                                                    xpt_links.stqe);
>
> +                               mtx_unlock(&xsoftc.xpt_lock);
>                                 continue;
>                         } else {
>                                 /*
>
> Scott
>
>
> Michael Jung wrote:
>> I was running 7.0-BETA4 and had problems with SEG_FAULT 12 in g_up so
>> I rebuilt a stock GENERIC kernel with these additional options:
>>
>>
>>
>> options     KDB
>>
>> option      DDB
>>
>> options     INVARIANTS
>>
>> options     INVARIANT_SUPPORT
>>
>> options     WITNESS
>>
>> options     DEBUG_LOCKS
>>
>> options     DEBUG_VFS_LOCKS
>>
>> options     DIAGNOSTIC
>>
>>
>>
>> Here is the build date:
>>
>>
>>
>> FreeBSD charon.confluentasp.com 7.0-BETA4 FreeBSD 7.0-BETA4 #6: Tue
>> Dec
>> 11 09:05:50 EST 2007
>> mikej_at_charon.confluentasp.com:/usr/obj/usr/src/sys/CHARON  i386
>>
>> (I cvsup'd source on this day just prior to kernel build -
>> tag=RELENG_7)
>>
>>
>>
>> Now when my SCSI array is plugged into the AHA-2944 controller the
>> kernel panics on boot.  If the array is not plugged
>>
>> into the controller the system boots and runs fine.
>>
>>
>>
>> I've read the handbook and tried to follow what posts I read as to
>> what would be of use in trouble shooting this.
>>
>> Please don't shoot the messenger. I'll be glad to supply and
>> additional information - this box is not in production
>>
>> so I'm willing to try anything.
>>
>>
>>
>> Just to note, the controller and array had been in production under
>> 6.2-RELEASE and prior versions and has been running
>>
>> for years without any problems.
>>
>>
>>
>> Thanks.
>>
>>
>>
>> --mikej
>>
>>
>>
>> KDB: debugger backends: ddb
>>
>> KDB: current backend: ddb
>>
>> Copyright (c) 1992-2007 The FreeBSD Project.
>>
>> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993,
>> 1994
>>
>>            The Regents of the University of California. All rights
>> reserved.
>>
>> FreeBSD is a registered trademark of The FreeBSD Foundation.
>>
>> FreeBSD 7.0-BETA4 #6: Tue Dec 11 09:05:50 EST 2007
>>
>>    mikej_at_charon.confluentasp.com:/usr/obj/usr/src/sys/CHARON
>>
>> WARNING: WITNESS option enabled, expect reduced performance.
>>
>> WARNING: DIAGNOSTIC option enabled, expect reduced performance.
>>
>> Timecounter "i8254" frequency 1193182 Hz quality 0
>>
>> CPU: Mobile AMD Sempron(tm) Processor 3000+ (1799.81-MHz 686-class
>> CPU)
>>
>>  Origin = "AuthenticAMD"  Id = 0x20fc2  Stepping = 2
>>
>>
>> Features 
>> =0x78bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PG
>> E, MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2>
>>
>>  Features2=0x1<SSE3>
>>
>>  AMD Features=0xc2500800<SYSCALL,NX,MMX+,FFXSR,3DNow!+,3DNow!>
>>
>>  AMD Features2=0x1<LAHF>
>>
>> real memory  = 1005453312 (958 MB)
>>
>> avail memory = 969691136 (924 MB)
>>
>> ACPI APIC Table: <VIAK8M AWRDACPI>
>>
>> WITNESS: spin lock intrcnt not in order list
>>
>> ioapic0 <Version 0.3> irqs 0-23 on motherboard
>>
>> kbd1 at kbdmux0
>>
>> ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413,
>> RF5413)
>>
>> acpi0: <VIAK8M AWRDACPI> on motherboard
>>
>> acpi0: [ITHREAD]
>>
>> acpi0: Power Button (fixed)
>>
>> acpi0: reservation of 0, a0000 (3) failed
>>
>> acpi0: reservation of 100000, 3bde0000 (3) failed
>>
>> Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
>>
>> acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on  
>> acpi0
>>
>> cpu0: <ACPI CPU> on acpi0
>>
>> powernow0: <PowerNow! K8> on cpu0
>>
>> acpi_button0: <Power Button> on acpi0
>>
>> acpi_button1: <Sleep Button> on acpi0
>>
>> pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
>>
>> pci0: <ACPI PCI bus> on pcib0
>>
>> agp0: <VIA 8380 host to PCI bridge> on hostb0
>>
>> pcib1: <PCI-PCI bridge> at device 1.0 on pci0
>>
>> pci1: <PCI bus> on pcib1
>>
>> vgapci0: <VGA-compatible display> mem
>> 0xf0000000-0xf3ffffff,0xf4000000-0xf4ffffff irq 16 at device 0.0 on
>> pci1
>>
>> pcib2: <PCI-PCI bridge> at device 8.0 on pci0
>>
>> pci2: <PCI bus> on pcib2
>>
>> ahc0: <Adaptec 4944 Ultra SCSI adapter> port 0xc000-0xc0ff mem
>> 0xf6013000-0xf6013fff irq 16 at device 4.0 on pci2
>>
>> ahc0: [ITHREAD]
>>
>> aic7880: Ultra Wide Channel A, SCSI Id=7, 16/253 SCBs
>>
>> ahc1: <Adaptec 4944 Ultra SCSI adapter> port 0xc400-0xc4ff mem
>> 0xf6010000-0xf6010fff irq 17 at device 5.0 on pci2
>>
>> ahc1: [ITHREAD]
>>
>> aic7880: Ultra Wide Channel A, SCSI Id=7, 16/253 SCBs
>>
>> ahc2: <Adaptec 4944 Ultra SCSI adapter> port 0xc800-0xc8ff mem
>> 0xf6011000-0xf6011fff irq 18 at device 6.0 on pci2
>>
>> ahc2: [ITHREAD]
>>
>> aic7880: Ultra Wide Channel A, SCSI Id=7, 16/253 SCBs
>>
>> ahc3: <Adaptec 4944 Ultra SCSI adapter> port 0xcc00-0xccff mem
>> 0xf6012000-0xf6012fff irq 19 at device 7.0 on pci2
>>
>> ahc3: [ITHREAD]
>>
>> aic7880: Ultra Wide Channel A, SCSI Id=7, 16/253 SCBs
>>
>> atapci0: <VIA 6420 SATA150 controller> port
>> 0xe100 
>> -0xe107,0xe700-0xe703,0xe800-0xe807,0xe900-0xe903,0xe000-0xe00f,
>> 0x d000-0xd0ff irq 20 at device 15.0 on pci0
>>
>> atapci0: [ITHREAD]
>>
>> ata2: <ATA channel 0> on atapci0
>>
>> ata2: [ITHREAD]
>>
>> ata3: <ATA channel 1> on atapci0
>>
>> ata3: [ITHREAD]
>>
>> atapci1: <VIA 8237 UDMA133 controller> port
>> 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xe200-0xe20f at device 15.1 on
>> pci0
>>
>> ata0: <ATA channel 0> on atapci1
>>
>> ata0: [ITHREAD]
>>
>> ata1: <ATA channel 1> on atapci1
>>
>> ata1: [ITHREAD]
>>
>> uhci0: <VIA 83C572 USB controller> port 0xe300-0xe31f irq 21 at  
>> device
>
>> 16.0 on pci0
>>
>> uhci0: [GIANT-LOCKED]
>>
>> uhci0: [ITHREAD]
>>
>> usb0: <VIA 83C572 USB controller> on uhci0
>>
>> usb0: USB revision 1.0
>>
>> uhub0: <VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb0
>>
>> uhub0: 2 ports with 2 removable, self powered
>>
>> uhci1: <VIA 83C572 USB controller> port 0xe400-0xe41f irq 21 at  
>> device
>> 16.1 on pci0
>>
>> uhci1: [GIANT-LOCKED]
>>
>> uhci1: [ITHREAD]
>>
>> usb1: <VIA 83C572 USB controller> on uhci1
>>
>> usb1: USB revision 1.0
>>
>> uhub1: <VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb1
>>
>> uhub1: 2 ports with 2 removable, self powered
>>
>> uhci2: <VIA 83C572 USB controller> port 0xe500-0xe51f irq 21 at  
>> device
>> 16.2 on pci0
>>
>> uhci2: [GIANT-LOCKED]
>>
>> uhci2: [ITHREAD]
>>
>> usb2: <VIA 83C572 USB controller> on uhci2
>>
>> usb2: USB revision 1.0
>>
>> uhub2: <VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb2
>>
>> uhub2: 2 ports with 2 removable, self powered
>>
>> uhci3: <VIA 83C572 USB controller> port 0xe600-0xe61f irq 21 at  
>> device
>> 16.3 on pci0
>>
>> uhci3: [GIANT-LOCKED]
>>
>> uhci3: [ITHREAD]
>>
>> usb3: <VIA 83C572 USB controller> on uhci3
>>
>> usb3: USB revision 1.0
>>
>> uhub3: <VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb3
>>
>> uhub3: 2 ports with 2 removable, self powered
>>
>> ehci0: <VIA VT6202 USB 2.0 controller> mem 0xf6100000-0xf61000ff irq
>> 21 at device 16.4 on pci0
>>
>> ehci0: [GIANT-LOCKED]
>>
>> ehci0: [ITHREAD]
>>
>> usb4: EHCI version 1.0
>>
>> usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3
>>
>> usb4: <VIA VT6202 USB 2.0 controller> on ehci0
>>
>> usb4: USB revision 2.0
>>
>> uhub4: <VIA EHCI root hub, class 9/0, rev 2.00/1.00, addr 1> on usb4
>>
>> uhub4: 8 ports with 8 removable, self powered
>>
>> isab0: <PCI-ISA bridge> at device 17.0 on pci0
>>
>> isa0: <ISA bus> on isab0
>>
>> pci0: <multimedia, audio> at device 17.5 (no driver attached)
>>
>> vr0: <VIA VT6102 Rhine II 10/100BaseTX> port 0xdc00-0xdcff mem
>> 0xf6101000-0xf61010ff irq 23 at device 18.0 on pci0
>>
>> vr0: Quirks: 0x0
>>
>> miibus0: <MII bus> on vr0
>>
>> rlphy0: <RTL8201L 10/100 media interface> PHY 1 on miibus0
>>
>> rlphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
>>
>> vr0: using obsoleted if_watchdog interface
>>
>> vr0: Ethernet address: 00:e0:4d:0a:1b:e6
>>
>> vr0: [ITHREAD]
>>
>> acpi_tz0: <Thermal Zone> on acpi0
>>
>> fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on
>> acpi0
>>
>> fdc0: [FILTER]
>>
>> fd0: <1440-KB 3.5" drive> on fdc0 drive 0
>>
>> sio0: configured irq 4 not in bitmap of probed irqs 0
>>
>> sio0: port may not be enabled
>>
>> sio0: configured irq 4 not in bitmap of probed irqs 0
>>
>> sio0: port may not be enabled
>>
>> sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10
>> on acpi0
>>
>> sio0: type 16550A, console
>>
>> sio0: [FILTER]
>>
>> atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
>>
>> atkbd0: <AT Keyboard> irq 1 on atkbdc0
>>
>> kbd0 at atkbd0
>>
>> atkbd0: [GIANT-LOCKED]
>>
>> atkbd0: [ITHREAD]
>>
>> psm0: <PS/2 Mouse> irq 12 on atkbdc0
>>
>> psm0: [GIANT-LOCKED]
>>
>> psm0: [ITHREAD]
>>
>> psm0: model IntelliMouse, device ID 3
>>
>> pmtimer0 on isa0
>>
>> orm0: <ISA Option ROM> at iomem 0xcc000-0xcc7ff pnpid ORM0000 on isa0
>>
>> ppc0: <Parallel port> at port 0x378-0x37f irq 7 on isa0
>>
>> ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode
>>
>> ppbus0: <Parallel port bus> on ppc0
>>
>> lpt0: <Printer> on ppbus0
>>
>> lpt0: Interrupt-driven port
>>
>> ppi0: <Parallel I/O> on ppbus0
>>
>> plip0: <PLIP network interface> on ppbus0
>>
>> ppc0: [GIANT-LOCKED]
>>
>> ppc0: [ITHREAD]
>>
>> sc0: <System console> at flags 0x100 on isa0
>>
>> sc0: VGA <16 virtual consoles, flags=0x300>
>>
>> sio1: configured irq 3 not in bitmap of probed irqs 0
>>
>> sio1: port may not be enabled
>>
>> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on
>> isa0
>>
>> Timecounter "TSC" frequency 1799808459 Hz quality 800
>>
>> Timecounters tick every 1.000 msec
>>
>> Waiting 5 seconds for SCSI devices to settle
>>
>> ad4: 381554MB <WDC WD4000AAKS-00TMA0 12.01C01> at ata2-master SATA150
>>
>> suspending ithread with the following locks held:
>>
>> exclusive sleep mutex XPT lock r = 0 (0xc0b883a4) locked _at_
>> /usr/src/sys/cam/cam_xpt.c:3852
>>
>> panic: witness_warn
>>
>> cpuid = 0
>>
>> KDB: enter: panic
>>
>> [thread pid 18 tid 100016 ]
>>
>> Stopped at      kdb_enter+0x32: leave
>>
>> db> ps
>>
>>  pid  ppid  pgrp   uid   state   wmesg     wchan    cmd
>>
>>   53     0     0     0  SL      waiting_ 0xc0c084cc [sctp_iterator]
>>
>>   52     0     0     0  WL                          [irq7: ppc0]
>>
>>   51     0     0     0  WL                          [irq12: psm0]
>>
>>   50     0     0     0  WL                          [irq1: atkbd0]
>>
>>   49     0     0     0  WL                          [swi0: sio]
>>
>>   48     0     0     0  SL      -        0xc401f23c [fdc0]
>>
>>   47     0     0     0  SL      cooling  0xc3fedad4 [acpi_cooling0]
>>
>>   46     0     0     0  SL      tzpoll   0xc0d85d40 [acpi_thermal]
>>
>>   45     0     0     0  WL                          [irq23: vr0]
>>
>>   44     0     0     0  SL      usbevt   0xc3fe4210 [usb4]
>>
>>   43     0     0     0  SL      usbevt   0xc3ffb210 [usb3]
>>
>>   42     0     0     0  SL      usbevt   0xc3ff5210 [usb2]
>>
>>   41     0     0     0  SL      usbevt   0xc3fee210 [usb1]
>>
>>   40     0     0     0  SL      usbtsk   0xc0bb7174 [usbtask-dr]
>>
>>   39     0     0     0  SL      usbtsk   0xc0bb7160 [usbtask-hc]
>>
>>   38     0     0     0  SL      usbevt   0xc3fc8210 [usb0]
>>
>>   37     0     0     0  WL                          [irq21: uhci0
>> uhci*]
>>
>>   36     0     0     0  WL                          [irq15: ata1]
>>
>>   35     0     0     0  WL                          [irq14: ata0]
>>
>> --More--
>>
>>
>>
>>   34     0     0     0  WL                          [irq20: atapci0]
>>
>>   33     0     0     0  SL      idle     0xc3ef1e00 [aic_recovery3]
>>
>>   32     0     0     0  RL                          [irq19: ahc3]
>>
>>   31     0     0     0  SL      idle     0xc3ef1e00 [aic_recovery3]
>>
>>   30     0     0     0  SL      idle     0xc3ef1400 [aic_recovery2]
>>
>>   29     0     0     0  RL                          [irq18: ahc2]
>>
>>   28     0     0     0  SL      idle     0xc3ef1400 [aic_recovery2]
>>
>>   27     0     0     0  SL      idle     0xc3ef0a00 [aic_recovery1]
>>
>>   26     0     0     0  RL                          [irq17: ahc1]
>>
>>   25     0     0     0  SL      idle     0xc3ef0a00 [aic_recovery1]
>>
>>   24     0     0     0  SL      idle     0xc3ef0c00 [aic_recovery0]
>>
>>   23     0     0     0  WL                          [irq16: ahc0]
>>
>>   22     0     0     0  SL      idle     0xc3ef0c00 [aic_recovery0]
>>
>>   21     0     0     0  WL                          [irq9: acpi0]
>>
>>   20     0     0     0  SL      -        0xc3ed9580 [thread taskq]
>>
>>   19     0     0     0  WL                          [swi5: +]
>>
>>   18     0     0     0  RL      CPU 0               [swi2: cambio]
>>
>>    9     0     0     0  SL      ccb_scan 0xc0b88374 [xpt_thrd]
>>
>>    8     0     0     0  SL      -        0xc3ed9900 [acpi_task_2]
>>
>>    7     0     0     0  SL      -        0xc3ed9900 [acpi_task_1]
>>
>> --More--
>>
>>
>>
>>    6     0     0     0  SL      -        0xc3ed9900 [acpi_task_0]
>>
>>    5     0     0     0  SL      -        0xc3ed9980 [kqueue taskq]
>>
>>   17     0     0     0  WL                          [swi6: task
> queue]
>>
>>   16     0     0     0  WL                          [swi6: Giant
> taskq]
>>
>>   15     0     0     0  SL      -        0xc0bb94d4 [yarrow]
>>
>>    4     0     0     0  SL      -        0xc0bb754c [g_down]
>>
>>    3     0     0     0  SL      -        0xc0bb7548 [g_up]
>>
>>    2     0     0     0  SL      -        0xc0bb7540 [g_event]
>>
>>   14     0     0     0  WL                          [swi3: vm]
>>
>>   13     0     0     0  RL                          [swi4: clock sio]
>>
>>   12     0     0     0  WL                          [swi1: net]
>>
>>   11     0     0     0  RL                          [idle: cpu0]
>>
>>    1     0     0     0  ?L                          [swapper]
>>
>>   10     0     0     0  SL      audit_wo 0xc0c11ad4 [audit]
>>
>>    0     0     0     0  SLs     conifhk  0xc0b599ec [swapper]
>>
>> db> show reg
>>
>> cs                0x20
>>
>> ds                0x28
>>
>> es                0x28
>>
>> fs                 0x8
>>
>> ss                0x28
>>
>> eax               0x12
>>
>> ecx         0xc1434bde
>>
>> edx                  0
>>
>> ebx              0x100
>>
>> esp         0xe175cc5c
>>
>> ebp         0xe175cc64
>>
>> esi                0x1
>>
>> edi         0xc3dea210
>>
>> eip         0xc0777ca2  kdb_enter+0x32
>>
>> efl            0x80282
>>
>> kdb_enter+0x32: leave
>>
>> db> show locks
>>
>> exclusive sleep mutex XPT lock r = 0 (0xc0b883a4) locked _at_
>> /usr/src/sys/cam/cam_xpt.c:3852
>>
>> db> show lockedvnods
>>
>> Locked vnodes
>>
>> db> bt
>>
>> Tracing pid 18 tid 100016 td 0xc3dea210
>>
>> kdb_enter(c0aa2af2,0,c0aa7718,e175cc9c,0,...) at kdb_enter+0x32
>>
>> panic(c0aa7718,e175ccd8,0,0,0,...) at panic+0x124
>>
>> witness_warn(2,0,c0a9f7d0,471,c3ed97e4,...) at witness_warn+0x19c
>>
>> ithread_loop(c3ed80d0,e175cd38,c0a9f445,2ea,c3ee42ac,...) at
>> ithread_loop+0x2ca
>>
>> fork_exit(c0735670,c3ed80d0,e175cd38) at fork_exit+0xb8
>>
>> fork_trampoline() at fork_trampoline+0x8
>>
>> --- trap 0, eip = 0, esp = 0xe175cd70, ebp = 0 ---
>>
>> db>
>>
>>
>> CONFIDENTIALITY NOTE: This message is intended only for the use of  
>> the
>
>> individual or entity to whom it is addressed and may contain
>> information that is privileged, confidential, and exempt from
>> disclosure under applicable law. If the reader of this message is not
>> the intended recipient, you are hereby notified that any
>> dissemination, distribution or copying of this communication is
>> strictly prohibited. If you have received this transmission in error,
>> please notify us by telephone at (502) 212-4001 or notify us at PAI ,
>> Dept. 99, 11857 Commonwealth Drive, Louisville, KY  40299.  Thank  
>> you.
>> _______________________________________________
>> freebsd-current_at_freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-current
>> To unsubscribe, send any mail to
> "freebsd-current-unsubscribe_at_freebsd.org"
>
> Scott,
>
> Any chance on getting this committed?  It's been working great but
> requires me to
> patch after every cvsup.
>
> Thanks.
>
> --mikej
>
> CONFIDENTIALITY NOTE: This message is intended only for the use
> of the individual or entity to whom it is addressed and may contain
> information that is privileged, confidential, and exempt from
> disclosure under applicable law. If the reader of this message is
> not the intended recipient, you are hereby notified that any
> dissemination, distribution or copying of this communication
> is strictly prohibited. If you have received this transmission
> in error, please notify us by telephone at (502) 212-4001 or
> notify us at PAI , Dept. 99, 11857 Commonwealth Drive,
> Louisville, KY  40299.  Thank you.
Received on Wed Jan 09 2008 - 05:34:34 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:39:25 UTC