7.0-BETA4 - witness_warn on boot

From: Michael Jung <mikej_at_paymentallianceintl.com>
Date: Fri, 14 Dec 2007 13:13:54 -0500
I was running 7.0-BETA4 and had problems with SEG_FAULT 12 in g_up so I
rebuilt a stock GENERIC kernel with these additional options:

 

options     KDB

option      DDB

options     INVARIANTS

options     INVARIANT_SUPPORT

options     WITNESS

options     DEBUG_LOCKS

options     DEBUG_VFS_LOCKS

options     DIAGNOSTIC

 

Here is the build date:

 

FreeBSD charon.confluentasp.com 7.0-BETA4 FreeBSD 7.0-BETA4 #6: Tue Dec
11 09:05:50 EST 2007
mikej_at_charon.confluentasp.com:/usr/obj/usr/src/sys/CHARON  i386

(I cvsup'd source on this day just prior to kernel build - tag=RELENG_7)

 

Now when my SCSI array is plugged into the AHA-2944 controller the
kernel panics on boot.  If the array is not plugged

into the controller the system boots and runs fine.

 

I've read the handbook and tried to follow what posts I read as to what
would be of use in trouble shooting this.  

Please don't shoot the messenger. I'll be glad to supply and additional
information - this box is not in production 

so I'm willing to try anything.

 

Just to note, the controller and array had been in production under
6.2-RELEASE and prior versions and has been running 

for years without any problems.

 

Thanks.

 

--mikej

 

KDB: debugger backends: ddb

KDB: current backend: ddb

Copyright (c) 1992-2007 The FreeBSD Project.

Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994

            The Regents of the University of California. All rights
reserved.

FreeBSD is a registered trademark of The FreeBSD Foundation.

FreeBSD 7.0-BETA4 #6: Tue Dec 11 09:05:50 EST 2007

    mikej_at_charon.confluentasp.com:/usr/obj/usr/src/sys/CHARON

WARNING: WITNESS option enabled, expect reduced performance.

WARNING: DIAGNOSTIC option enabled, expect reduced performance.

Timecounter "i8254" frequency 1193182 Hz quality 0

CPU: Mobile AMD Sempron(tm) Processor 3000+ (1799.81-MHz 686-class CPU)

  Origin = "AuthenticAMD"  Id = 0x20fc2  Stepping = 2

 
Features=0x78bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,
MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2>

  Features2=0x1<SSE3>

  AMD Features=0xc2500800<SYSCALL,NX,MMX+,FFXSR,3DNow!+,3DNow!>

  AMD Features2=0x1<LAHF>

real memory  = 1005453312 (958 MB)

avail memory = 969691136 (924 MB)

ACPI APIC Table: <VIAK8M AWRDACPI>

WITNESS: spin lock intrcnt not in order list

ioapic0 <Version 0.3> irqs 0-23 on motherboard

kbd1 at kbdmux0

ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413,
RF5413)

acpi0: <VIAK8M AWRDACPI> on motherboard

acpi0: [ITHREAD]

acpi0: Power Button (fixed)

acpi0: reservation of 0, a0000 (3) failed

acpi0: reservation of 100000, 3bde0000 (3) failed

Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000

acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0

cpu0: <ACPI CPU> on acpi0

powernow0: <PowerNow! K8> on cpu0

acpi_button0: <Power Button> on acpi0

acpi_button1: <Sleep Button> on acpi0

pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0

pci0: <ACPI PCI bus> on pcib0

agp0: <VIA 8380 host to PCI bridge> on hostb0

pcib1: <PCI-PCI bridge> at device 1.0 on pci0

pci1: <PCI bus> on pcib1

vgapci0: <VGA-compatible display> mem
0xf0000000-0xf3ffffff,0xf4000000-0xf4ffffff irq 16 at device 0.0 on pci1

pcib2: <PCI-PCI bridge> at device 8.0 on pci0

pci2: <PCI bus> on pcib2

ahc0: <Adaptec 4944 Ultra SCSI adapter> port 0xc000-0xc0ff mem
0xf6013000-0xf6013fff irq 16 at device 4.0 on pci2

ahc0: [ITHREAD]

aic7880: Ultra Wide Channel A, SCSI Id=7, 16/253 SCBs

ahc1: <Adaptec 4944 Ultra SCSI adapter> port 0xc400-0xc4ff mem
0xf6010000-0xf6010fff irq 17 at device 5.0 on pci2

ahc1: [ITHREAD]

aic7880: Ultra Wide Channel A, SCSI Id=7, 16/253 SCBs

ahc2: <Adaptec 4944 Ultra SCSI adapter> port 0xc800-0xc8ff mem
0xf6011000-0xf6011fff irq 18 at device 6.0 on pci2

ahc2: [ITHREAD]

aic7880: Ultra Wide Channel A, SCSI Id=7, 16/253 SCBs

ahc3: <Adaptec 4944 Ultra SCSI adapter> port 0xcc00-0xccff mem
0xf6012000-0xf6012fff irq 19 at device 7.0 on pci2

ahc3: [ITHREAD]

aic7880: Ultra Wide Channel A, SCSI Id=7, 16/253 SCBs

atapci0: <VIA 6420 SATA150 controller> port
0xe100-0xe107,0xe700-0xe703,0xe800-0xe807,0xe900-0xe903,0xe000-0xe00f,0x
d000-0xd0ff irq 20 at device 15.0 on pci0

atapci0: [ITHREAD]

ata2: <ATA channel 0> on atapci0

ata2: [ITHREAD]

ata3: <ATA channel 1> on atapci0

ata3: [ITHREAD]

atapci1: <VIA 8237 UDMA133 controller> port
0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xe200-0xe20f at device 15.1 on pci0

ata0: <ATA channel 0> on atapci1

ata0: [ITHREAD]

ata1: <ATA channel 1> on atapci1

ata1: [ITHREAD]

uhci0: <VIA 83C572 USB controller> port 0xe300-0xe31f irq 21 at device
16.0 on pci0

uhci0: [GIANT-LOCKED]

uhci0: [ITHREAD]

usb0: <VIA 83C572 USB controller> on uhci0

usb0: USB revision 1.0

uhub0: <VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb0

uhub0: 2 ports with 2 removable, self powered

uhci1: <VIA 83C572 USB controller> port 0xe400-0xe41f irq 21 at device
16.1 on pci0

uhci1: [GIANT-LOCKED]

uhci1: [ITHREAD]

usb1: <VIA 83C572 USB controller> on uhci1

usb1: USB revision 1.0

uhub1: <VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb1

uhub1: 2 ports with 2 removable, self powered

uhci2: <VIA 83C572 USB controller> port 0xe500-0xe51f irq 21 at device
16.2 on pci0

uhci2: [GIANT-LOCKED]

uhci2: [ITHREAD]

usb2: <VIA 83C572 USB controller> on uhci2

usb2: USB revision 1.0

uhub2: <VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb2

uhub2: 2 ports with 2 removable, self powered

uhci3: <VIA 83C572 USB controller> port 0xe600-0xe61f irq 21 at device
16.3 on pci0

uhci3: [GIANT-LOCKED]

uhci3: [ITHREAD]

usb3: <VIA 83C572 USB controller> on uhci3

usb3: USB revision 1.0

uhub3: <VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb3

uhub3: 2 ports with 2 removable, self powered

ehci0: <VIA VT6202 USB 2.0 controller> mem 0xf6100000-0xf61000ff irq 21
at device 16.4 on pci0

ehci0: [GIANT-LOCKED]

ehci0: [ITHREAD]

usb4: EHCI version 1.0

usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3

usb4: <VIA VT6202 USB 2.0 controller> on ehci0

usb4: USB revision 2.0

uhub4: <VIA EHCI root hub, class 9/0, rev 2.00/1.00, addr 1> on usb4

uhub4: 8 ports with 8 removable, self powered

isab0: <PCI-ISA bridge> at device 17.0 on pci0

isa0: <ISA bus> on isab0

pci0: <multimedia, audio> at device 17.5 (no driver attached)

vr0: <VIA VT6102 Rhine II 10/100BaseTX> port 0xdc00-0xdcff mem
0xf6101000-0xf61010ff irq 23 at device 18.0 on pci0

vr0: Quirks: 0x0

miibus0: <MII bus> on vr0

rlphy0: <RTL8201L 10/100 media interface> PHY 1 on miibus0

rlphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto

vr0: using obsoleted if_watchdog interface

vr0: Ethernet address: 00:e0:4d:0a:1b:e6

vr0: [ITHREAD]

acpi_tz0: <Thermal Zone> on acpi0

fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on
acpi0

fdc0: [FILTER]

fd0: <1440-KB 3.5" drive> on fdc0 drive 0

sio0: configured irq 4 not in bitmap of probed irqs 0

sio0: port may not be enabled

sio0: configured irq 4 not in bitmap of probed irqs 0

sio0: port may not be enabled

sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on
acpi0

sio0: type 16550A, console

sio0: [FILTER]

atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0

atkbd0: <AT Keyboard> irq 1 on atkbdc0

kbd0 at atkbd0

atkbd0: [GIANT-LOCKED]

atkbd0: [ITHREAD]

psm0: <PS/2 Mouse> irq 12 on atkbdc0

psm0: [GIANT-LOCKED]

psm0: [ITHREAD]

psm0: model IntelliMouse, device ID 3

pmtimer0 on isa0

orm0: <ISA Option ROM> at iomem 0xcc000-0xcc7ff pnpid ORM0000 on isa0

ppc0: <Parallel port> at port 0x378-0x37f irq 7 on isa0

ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode

ppbus0: <Parallel port bus> on ppc0

lpt0: <Printer> on ppbus0

lpt0: Interrupt-driven port

ppi0: <Parallel I/O> on ppbus0

plip0: <PLIP network interface> on ppbus0

ppc0: [GIANT-LOCKED]

ppc0: [ITHREAD]

sc0: <System console> at flags 0x100 on isa0

sc0: VGA <16 virtual consoles, flags=0x300>

sio1: configured irq 3 not in bitmap of probed irqs 0

sio1: port may not be enabled

vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on
isa0

Timecounter "TSC" frequency 1799808459 Hz quality 800

Timecounters tick every 1.000 msec

Waiting 5 seconds for SCSI devices to settle

ad4: 381554MB <WDC WD4000AAKS-00TMA0 12.01C01> at ata2-master SATA150

suspending ithread with the following locks held:

exclusive sleep mutex XPT lock r = 0 (0xc0b883a4) locked _at_
/usr/src/sys/cam/cam_xpt.c:3852

panic: witness_warn

cpuid = 0

KDB: enter: panic

[thread pid 18 tid 100016 ]

Stopped at      kdb_enter+0x32: leave

db> ps

  pid  ppid  pgrp   uid   state   wmesg     wchan    cmd

   53     0     0     0  SL      waiting_ 0xc0c084cc [sctp_iterator]

   52     0     0     0  WL                          [irq7: ppc0]

   51     0     0     0  WL                          [irq12: psm0]

   50     0     0     0  WL                          [irq1: atkbd0]

   49     0     0     0  WL                          [swi0: sio]

   48     0     0     0  SL      -        0xc401f23c [fdc0]

   47     0     0     0  SL      cooling  0xc3fedad4 [acpi_cooling0]

   46     0     0     0  SL      tzpoll   0xc0d85d40 [acpi_thermal]

   45     0     0     0  WL                          [irq23: vr0]

   44     0     0     0  SL      usbevt   0xc3fe4210 [usb4]

   43     0     0     0  SL      usbevt   0xc3ffb210 [usb3]

   42     0     0     0  SL      usbevt   0xc3ff5210 [usb2]

   41     0     0     0  SL      usbevt   0xc3fee210 [usb1]

   40     0     0     0  SL      usbtsk   0xc0bb7174 [usbtask-dr]

   39     0     0     0  SL      usbtsk   0xc0bb7160 [usbtask-hc]

   38     0     0     0  SL      usbevt   0xc3fc8210 [usb0]

   37     0     0     0  WL                          [irq21: uhci0
uhci*]

   36     0     0     0  WL                          [irq15: ata1]

   35     0     0     0  WL                          [irq14: ata0]

--More--

        

   34     0     0     0  WL                          [irq20: atapci0]

   33     0     0     0  SL      idle     0xc3ef1e00 [aic_recovery3]

   32     0     0     0  RL                          [irq19: ahc3]

   31     0     0     0  SL      idle     0xc3ef1e00 [aic_recovery3]

   30     0     0     0  SL      idle     0xc3ef1400 [aic_recovery2]

   29     0     0     0  RL                          [irq18: ahc2]

   28     0     0     0  SL      idle     0xc3ef1400 [aic_recovery2]

   27     0     0     0  SL      idle     0xc3ef0a00 [aic_recovery1]

   26     0     0     0  RL                          [irq17: ahc1]

   25     0     0     0  SL      idle     0xc3ef0a00 [aic_recovery1]

   24     0     0     0  SL      idle     0xc3ef0c00 [aic_recovery0]

   23     0     0     0  WL                          [irq16: ahc0]

   22     0     0     0  SL      idle     0xc3ef0c00 [aic_recovery0]

   21     0     0     0  WL                          [irq9: acpi0]

   20     0     0     0  SL      -        0xc3ed9580 [thread taskq]

   19     0     0     0  WL                          [swi5: +]

   18     0     0     0  RL      CPU 0               [swi2: cambio]

    9     0     0     0  SL      ccb_scan 0xc0b88374 [xpt_thrd]

    8     0     0     0  SL      -        0xc3ed9900 [acpi_task_2]

    7     0     0     0  SL      -        0xc3ed9900 [acpi_task_1]

--More--

        

    6     0     0     0  SL      -        0xc3ed9900 [acpi_task_0]

    5     0     0     0  SL      -        0xc3ed9980 [kqueue taskq]

   17     0     0     0  WL                          [swi6: task queue]

   16     0     0     0  WL                          [swi6: Giant taskq]

   15     0     0     0  SL      -        0xc0bb94d4 [yarrow]

    4     0     0     0  SL      -        0xc0bb754c [g_down]

    3     0     0     0  SL      -        0xc0bb7548 [g_up]

    2     0     0     0  SL      -        0xc0bb7540 [g_event]

   14     0     0     0  WL                          [swi3: vm]

   13     0     0     0  RL                          [swi4: clock sio]

   12     0     0     0  WL                          [swi1: net]

   11     0     0     0  RL                          [idle: cpu0]

    1     0     0     0  ?L                          [swapper]

   10     0     0     0  SL      audit_wo 0xc0c11ad4 [audit]

    0     0     0     0  SLs     conifhk  0xc0b599ec [swapper]

db> show reg

cs                0x20

ds                0x28

es                0x28

fs                 0x8

ss                0x28

eax               0x12

ecx         0xc1434bde

edx                  0

ebx              0x100

esp         0xe175cc5c

ebp         0xe175cc64

esi                0x1

edi         0xc3dea210

eip         0xc0777ca2  kdb_enter+0x32

efl            0x80282

kdb_enter+0x32: leave

db> show locks

exclusive sleep mutex XPT lock r = 0 (0xc0b883a4) locked _at_
/usr/src/sys/cam/cam_xpt.c:3852

db> show lockedvnods

Locked vnodes

db> bt

Tracing pid 18 tid 100016 td 0xc3dea210

kdb_enter(c0aa2af2,0,c0aa7718,e175cc9c,0,...) at kdb_enter+0x32

panic(c0aa7718,e175ccd8,0,0,0,...) at panic+0x124

witness_warn(2,0,c0a9f7d0,471,c3ed97e4,...) at witness_warn+0x19c

ithread_loop(c3ed80d0,e175cd38,c0a9f445,2ea,c3ee42ac,...) at
ithread_loop+0x2ca

fork_exit(c0735670,c3ed80d0,e175cd38) at fork_exit+0xb8

fork_trampoline() at fork_trampoline+0x8

--- trap 0, eip = 0, esp = 0xe175cd70, ebp = 0 ---

db>


CONFIDENTIALITY NOTE: This message is intended only for the use
of the individual or entity to whom it is addressed and may contain 
information that is privileged, confidential, and exempt from 
disclosure under applicable law. If the reader of this message is 
not the intended recipient, you are hereby notified that any 
dissemination, distribution or copying of this communication 
is strictly prohibited. If you have received this transmission 
in error, please notify us by telephone at (502) 212-4001 or 
notify us at PAI , Dept. 99, 11857 Commonwealth Drive, 
Louisville, KY  40299.  Thank you.
Received on Fri Dec 14 2007 - 17:57:36 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:39:24 UTC