zfs woes

From: Duncan Young <duncan.young_at_pobox.com>
Date: Fri, 11 Sep 2009 08:47:16 +1000
Hi all,

Again its happened (my need to restore my main zpool from backups).

I have been having regular crashes (only recently been getting crash
dumps though), and in the the process
of upgrading to 7.1-STABLE, upgrading zpools (now 13) and zfs (now 3), I
now have a problem with
my main data zpool.

I was having these problems before the upgrade.

After zpool status saying that I had corrupted files (actually a zfs
filesystem), I tried destroying it (the zfs filesystem), in the hope
that this
may make my problems go away).  The machine hung.

zpool scrub works, reporting 16 errors.
zpool status -v crashes.
starting multiuser hangs (probably on mounting the dud fs).
I can start single user, and I am currently additional backups of the
filesytems I can access (most off the corrupted zpool).
Any accessing the corrupted zfs filesystem crashes the machine.

Now I think it's fair enough that the filesystem is corrupted (I still
see a need for some type of fsck), but I don't see that the
machine should crash because of it. Any sugestions?

Script started on Fri Sep 11 17:22:23 2009
# uname -a
FreeBSD  7.2-STABLE FreeBSD 7.2-STABLE #8: Fri Sep 11 04:14:39 EST
2009     root_at_:/usr/obj/usr/src/sys/TRIPLE0  amd64
# cd /usr/src/sys/TRIPLE0
# kgdb ./kernel.debug /var/crash/vmcore/.25
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...

Unread portion of the kernel message buffer:
Copyright (c) 1992-2009 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
    The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 7.2-STABLE #8: Fri Sep 11 04:14:39 EST 2009
    root_at_:/usr/obj/usr/src/sys/TRIPLE0
module_register: module g_label already exists!
Module g_label failed to register: 17
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: AMD Phenom(tm) II X4 810 Processor (2608.81-MHz K8-class CPU)
  Origin = "AuthenticAMD"  Id = 0x100f42  Stepping = 2

Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x802009<SSE3,MON,CX16,POPCNT>
  AMD
Features=0xee500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM,3DNow!+,3DNow!>
  AMD
Features2=0x37ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,SKINIT,WDT>
  TSC: P-state invariant
  Cores per package: 4
usable memory = 8441266176 (8050 MB)
avail memory  = 8131702784 (7754 MB)
ACPI APIC Table: <072709 APIC1045>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
cpu0 (BSP): APIC ID:  0
cpu1 (AP): APIC ID:  1
cpu2 (AP): APIC ID:  2
cpu3 (AP): APIC ID:  3
This module (opensolaris) contains code covered by the
Common Development and Distribution License (CDDL)
see http://opensolaris.org/os/licensing/opensolaris_license/
ACPI Warning (tbfadt-0505): Optional field "Pm2ControlBlock" has zero
address or length:        0       0/1 [20070320]
ioapic0 <Version 2.1> irqs 0-23 on motherboard
kbd1 at kbdmux0
acpi0: <072709 XSDT1045> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
acpi0: reservation of fee00000, 1000 (3) failed
acpi0: reservation of ffb80000, 80000 (3) failed
acpi0: reservation of fec10000, 20 (3) failed
acpi0: reservation of 0, a0000 (3) failed
acpi0: reservation of 100000, dff00000 (3) failed
ACPI HPET table warning: Sequence is non-zero (2)
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <32-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on
acpi0
Timecounter "HPET" frequency 14318180 Hz quality 900
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
vgapci0: <VGA-compatible display> port 0xc000-0xc0ff mem
0xf0000000-0xf7ffffff,0xfbce0000-0xfbceffff,0xfbb00000-0xfbbfffff irq 18
at device 5.0 on pci1
hdac0: <ATI RS780 High Definition Audio Controller> mem
0xfbcfc000-0xfbcfffff irq 19 at device 5.1 on pci1
hdac0: HDA Driver Revision: 20090624_0136
hdac0: [ITHREAD]
pcib2: <ACPI PCI-PCI bridge> irq 16 at device 4.0 on pci0
pci2: <ACPI PCI bus> on pcib2
hptrr0: <sx508x> port 0xd800-0xd8ff mem 0xfbe00000-0xfbefffff irq 16 at
device 0.0 on pci2
hptrr: adapter at PCI 2:0:0, IRQ 16
pcib3: <ACPI PCI-PCI bridge> irq 17 at device 5.0 on pci0
pci3: <ACPI PCI bus> on pcib3
em0: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xec00-0xec1f mem
0xfbfe0000-0xfbffffff,0xfbf00000-0xfbf7ffff,0xfbfdc000-0xfbfdffff irq 17
at device 0.0 on pci3
em0: Using MSIX interrupts
em0: [ITHREAD]
em0: [ITHREAD]
em0: [ITHREAD]
em0: Ethernet address: 00:1b:21:3d:eb:e7
atapci0: <ATI AHCI controller> port
0xb000-0xb007,0xa000-0xa003,0x9000-0x9007,0x8000-0x8003,0x7000-0x700f
mem 0xfbaffc00-0xfbafffff irq 22 at device 17.0 on pci0
atapci0: [ITHREAD]
atapci0: AHCI Version 01.10 controller with 6 ports detected
ata2: <ATA channel 0> on atapci0
ata2: [ITHREAD]
ata3: <ATA channel 1> on atapci0
ata3: [ITHREAD]
ata4: <ATA channel 2> on atapci0
ata4: [ITHREAD]
ata5: <ATA channel 3> on atapci0
ata5: [ITHREAD]
ata6: <ATA channel 4> on atapci0
ata6: [ITHREAD]
ata7: <ATA channel 5> on atapci0
ata7: [ITHREAD]
ohci0: <OHCI (generic) USB controller> mem 0xfbafd000-0xfbafdfff irq 16
at device 18.0 on pci0
ohci0: [GIANT-LOCKED]
ohci0: [ITHREAD]
usb0: OHCI version 1.0, legacy support
usb0: <OHCI (generic) USB controller> on ohci0
usb0: USB revision 1.0
uhub0: <ATI OHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb0
uhub0: 3 ports with 3 removable, self powered
ohci1: <OHCI (generic) USB controller> mem 0xfbafe000-0xfbafefff irq 16
at device 18.1 on pci0
ohci1: [GIANT-LOCKED]
ohci1: [ITHREAD]
usb1: OHCI version 1.0, legacy support
usb1: <OHCI (generic) USB controller> on ohci1
usb1: USB revision 1.0
uhub1: <ATI OHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb1
uhub1: 3 ports with 3 removable, self powered
ehci0: <EHCI (generic) USB 2.0 controller> mem 0xfbaff800-0xfbaff8ff irq
17 at device 18.2 on pci0
ehci0: [GIANT-LOCKED]
ehci0: [ITHREAD]
usb2: EHCI version 1.0
usb2: companion controllers, 3 ports each: usb0 usb1
usb2: <EHCI (generic) USB 2.0 controller> on ehci0
usb2: USB revision 2.0
uhub2: <ATI EHCI root hub, class 9/0, rev 2.00/1.00, addr 1> on usb2
uhub2: 6 ports with 6 removable, self powered
ohci2: <OHCI (generic) USB controller> mem 0xfbafb000-0xfbafbfff irq 18
at device 19.0 on pci0
ohci2: [GIANT-LOCKED]
ohci2: [ITHREAD]
usb3: OHCI version 1.0, legacy support
usb3: <OHCI (generic) USB controller> on ohci2
usb3: USB revision 1.0
uhub3: <ATI OHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb3
uhub3: 3 ports with 3 removable, self powered
ohci3: <OHCI (generic) USB controller> mem 0xfbafc000-0xfbafcfff irq 18
at device 19.1 on pci0
ohci3: [GIANT-LOCKED]
ohci3: [ITHREAD]
usb4: OHCI version 1.0, legacy support
usb4: <OHCI (generic) USB controller> on ohci3
usb4: USB revision 1.0
uhub4: <ATI OHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb4
uhub4: 3 ports with 3 removable, self powered
ehci1: <EHCI (generic) USB 2.0 controller> mem 0xfbaff400-0xfbaff4ff irq
19 at device 19.2 on pci0
ehci1: [GIANT-LOCKED]
ehci1: [ITHREAD]
usb5: EHCI version 1.0
usb5: companion controllers, 3 ports each: usb3 usb4
usb5: <EHCI (generic) USB 2.0 controller> on ehci1
usb5: USB revision 2.0
uhub5: <ATI EHCI root hub, class 9/0, rev 2.00/1.00, addr 1> on usb5
uhub5: 6 ports with 6 removable, self powered
pci0: <serial bus, SMBus> at device 20.0 (no driver attached)
atapci1: <ATI IXP700 UDMA133 controller> port
0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xff00-0xff0f at device 20.1 on pci0
ata0: <ATA channel 0> on atapci1
ata0: [ITHREAD]
hdac1: <ATI SB600 High Definition Audio Controller> mem
0xfbaf4000-0xfbaf7fff irq 16 at device 20.2 on pci0
hdac1: HDA Driver Revision: 20090624_0136
hdac1: [ITHREAD]
isab0: <PCI-ISA bridge> at device 20.3 on pci0
isa0: <ISA bus> on isab0
pcib4: <ACPI PCI-PCI bridge> at device 20.4 on pci0
pci4: <ACPI PCI bus> on pcib4
ohci4: <OHCI (generic) USB controller> mem 0xfbafa000-0xfbafafff irq 18
at device 20.5 on pci0
ohci4: [GIANT-LOCKED]
ohci4: [ITHREAD]
usb6: OHCI version 1.0, legacy support
usb6: <OHCI (generic) USB controller> on ohci4
usb6: USB revision 1.0
uhub6: <ATI OHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb6
uhub6: 2 ports with 2 removable, self powered
acpi_button0: <Power Button> on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: [ITHREAD]
psm0: model MouseMan+, device ID 0
acpi_aiboost0: <ASUStek AIBOOSTER> on acpi0
sio0: configured irq 4 not in bitmap of probed irqs 0
sio0: port may not be enabled
sio0: configured irq 4 not in bitmap of probed irqs 0
sio0: port may not be enabled
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on
acpi0
sio0: type 16550A
sio0: [FILTER]
fdc0: <floppy drive controller (FDE)> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2
on acpi0
fdc0: [FILTER]
cpu0: <ACPI CPU> on acpi0
acpi_throttle0: <ACPI CPU Throttling> on cpu0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
cryptosoft0: <software crypto> on motherboard
orm0: <ISA Option ROM> at iomem 0xd7800-0xd87ff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ppc0: cannot reserve I/O port range
sio1: configured irq 3 not in bitmap of probed irqs 0
sio1: port may not be enabled
ulpt0: <Samsung Electronics Co., Ltd. Samsung Laser Printer ML-1200
Series, class 0/0, rev 1.10/1.00, addr 2> on uhub0
ulpt0: using bi-directional mode
WARNING: ZFS is considered to be an experimental feature in FreeBSD.
Timecounters tick every 1.000 msec
vboxdrv: fAsync=0 offMin=0x12c offMax=0x219
hptrr: start channel [0,0]
hptrr: start channel [0,1]
hptrr: start channel [0,2]
hptrr: start channel [0,3]
hptrr: [0 0] Start channel soft reset.
hptrr: [0 2] Start channel soft reset.
hptrr: [0 3] Start channel soft reset.
hptrr: channel [0,0] started successfully
hptrr: channel [0,2] started successfully
hptrr: channel [0,3] started successfully
hptrr: [0 1] Failed to perform channel hard reset.
hptrr0: [GIANT-LOCKED]
hptrr0: [ITHREAD]
The GEOM class LABEL is already loaded.
ZFS filesystem version 13
ZFS storage pool version 13
da0 at hptrr0 bus 0 target 0 lun 0
da0: <HPT DISK 0_0 4.00> Fixed Direct Access SCSI-0 device
da1 at hptrr0 bus 0 target 1 lun 0
da1: <HPT DISK 0_1 4.00> Fixed Direct Access SCSI-0 device
da2 at hptrr0 bus 0 target 2 lun 0
da2: <HPT DISK 0_2 4.00> Fixed Direct Access SCSI-0 device
acd0: DVDR <HL-DT-ST DVDRAM GSA-4163B/A104> at ata0-master PIO4
ad4: 476940MB <WDC WD5000AAKS-00C8A0 12.01C02> at ata2-master SATA300
acd0: FAILURE - INQUIRY ILLEGAL REQUEST asc=0x24 ascq=0x00 sks=0x40 0x00
0x01
(probe0:ata0:0:0:0): TEST UNIT READY. CDB: 0 0 0 0 0 0
(probe0:ata0:0:0:0): CAM Status: SCSI Status Error
(probe0:ata0:0:0:0): SCSI Status: Check Condition
(probe0:ata0:0:0:0): NOT READY csi:0,0,bb,0 asc:3a,0
(probe0:ata0:0:0:0): Medium not present
(probe0:ata0:0:0:0): Unretryable error
cd0 at ata0 bus 0 target 0 lun 0
cd0: <HL-DT-ST DVDRAM GSA-4163B A104> Removable CD-ROM SCSI-0 device
cd0: 16.000MB/s transfers
cd0: Attempt to query device size failed: NOT READY, Medium not present
ad6: 953869MB <WDC WD10EADS-00L5B1 01.01A01> at ata3-master SATA300
ad10: 953869MB <WDC WD10EACS-00D6B0 01.01A01> at ata5-master SATA300
ad12: 476940MB <WDC WD5000AACS-00ZUB0 01.01B01> at ata6-master SATA300
ad14: 953869MB <Seagate ST31000528AS CC44> at ata7-master SATA300
hdac0: HDA Codec #0: ATI RS690/780 HDMI
pcm0: <HDA ATI RS690/780 HDMI PCM #0 Digital> at cad 0 nid 1 on hdac0
hdac1: HDA Codec #0: VIA VT1708S_0
pcm1: <HDA VIA VT1708S_0 PCM #0 Analog> at cad 0 nid 1 on hdac1
pcm2: <HDA VIA VT1708S_0 PCM #1 Digital> at cad 0 nid 1 on hdac1
pcm3: <HDA VIA VT1708S_0 PCM #2 Digital> at cad 0 nid 1 on hdac1
SMP: AP CPU #2 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #1 Launched!
Trying to mount root from zfs:rootzfs
<118>Loading configuration files.
<118>kernel dumps on /dev/ad4s1b
<118>Entropy harvesting:
<118> interrupts
<118> ethernet
<118> point_to_point
<118> kickstart
<118>.
GEOM_ELI: Device label/swapB.eli created.
GEOM_ELI: Encryption: AES-CBC 256
GEOM_ELI:     Crypto: software
<118>swapon: adding /dev/label/swapB.eli as swap device
<118>/etc/rc: WARNING: Use of the early.sh script is deprecated
<118>/etc/rc: WARNING: Please use a new-style rc.d script instead
<118>/etc/rc: WARNING: See rc(8) for more information
<118>gmirror:
<118>No such device: swap_AB.
<118>
<118>Starting file system checks:
<118>/dev/label/bstrapA: FILE SYSTEM CLEAN; SKIPPING CHECKS
<118>/dev/label/bstrapA: clean, 303581 free (261 frags, 37915 blocks,
0.1% fragmentation)
<118>/dev/label/bstrapB: FILE SYSTEM CLEAN; SKIPPING CHECKS
<118>/dev/label/bstrapB: clean, 303502 free (318 frags, 37898 blocks,
0.1% fragmentation)
<118>Setting hostuuid: e039c626-5f54-da11-8e5a-e6b6a8a23cf0.
<118>Setting hostid: 0xd449202f.
<118>Mounting local file systems:
<118>.


Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 01
fault virtual address    = 0x50
fault code        = supervisor read data, page not present
instruction pointer    = 0x8:0xffffffff80ad7a5e
stack pointer            = 0x10:0xffffff80a5b2f620
frame pointer            = 0x10:0xffffff80a5b2f6a0
code segment        = base 0x0, limit 0xfffff, type 0x1b
            = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags    = interrupt enabled, resume, IOPL = 0
current process        = 314 (txg_thread_enter)
trap number        = 12
panic: page fault
cpuid = 1
Uptime: 2m11s
Physical memory: 8050 MB
Dumping 1553 MB: 1538 1522 1506 1490 1474 1458 1442 1426 1410 1394 1378
1362 1346 1330 1314 1298 1282 1266 1250 1234 1218 1202 1186 1170 1154
1138 1122 1106 1090 1074 1058 1042 1026 1010 994 978 962 946 930 914 898
882 866 850 834 818 802 786 770 754 738 722 706 690 674 658 642 626 610
594 578 562 546 530 514 498 482 466 450 434 418 402 386 370 354 338 322
306 290 274 258 242 226 210 194 178 162 146 130 114 98 82 66 50 34 18 2

Reading symbols from /boot/kernel/zfs.ko...Reading symbols from
/bootdirA/boot/kernel/zfs.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/zfs.ko
Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from
/bootdirA/boot/kernel/opensolaris.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/opensolaris.ko
Reading symbols from /boot/kernel/geom_eli.ko...Reading symbols from
/bootdirA/boot/kernel/geom_eli.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/geom_eli.ko
Reading symbols from /boot/kernel/crypto.ko...Reading symbols from
/bootdirA/boot/kernel/crypto.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/crypto.ko
Reading symbols from /boot/kernel/zlib.ko...Reading symbols from
/bootdirA/boot/kernel/zlib.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/zlib.ko
Reading symbols from /boot/kernel/geom_label.ko...Reading symbols from
/bootdirA/boot/kernel/geom_label.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/geom_label.ko
Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from
/bootdirA/boot/kernel/geom_mirror.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/geom_mirror.ko
Reading symbols from /boot/kernel/snd_hda.ko...Reading symbols from
/bootdirA/boot/kernel/snd_hda.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/snd_hda.ko
Reading symbols from /boot/kernel/sound.ko...Reading symbols from
/bootdirA/boot/kernel/sound.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/sound.ko
Reading symbols from /boot/kernel/cd9660_iconv.ko...Reading symbols from
/bootdirA/boot/kernel/cd9660_iconv.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/cd9660_iconv.ko
Reading symbols from /boot/kernel/libiconv.ko...Reading symbols from
/bootdirA/boot/kernel/libiconv.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/libiconv.ko
Reading symbols from /boot/kernel/aio.ko...Reading symbols from
/bootdirA/boot/kernel/aio.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/aio.ko
Reading symbols from /boot/modules/vboxdrv.ko...done.
Loaded symbols for /boot/modules/vboxdrv.ko
Reading symbols from /boot/kernel/sem.ko...Reading symbols from
/bootdirA/boot/kernel/sem.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/sem.ko
#0  doadump () at pcpu.h:195
195        __asm __volatile("movq %%gs:0,%0" : "=r" (td));
(kgdb) backtrace
#0  doadump () at pcpu.h:195
#1  0x0000000000000004 in ?? ()
#2  0xffffffff803aab90 in boot (howto=260)
    at /usr/src/sys/kern/kern_shutdown.c:418
#3  0xffffffff803aafb2 in panic (fmt=0x104 <Address 0x104 out of bounds>)
    at /usr/src/sys/kern/kern_shutdown.c:574
#4  0xffffffff80651d03 in trap_fatal (frame=0xffffff0009572390,
eva=Variable "eva" is not available.
)
    at /usr/src/sys/amd64/amd64/trap.c:770
#5  0xffffffff806520d5 in trap_pfault (frame=0xffffff80a5b2f570, usermode=0)
    at /usr/src/sys/amd64/amd64/trap.c:686
#6  0xffffffff80652a63 in trap (frame=0xffffff80a5b2f570)
    at /usr/src/sys/amd64/amd64/trap.c:457
#7  0xffffffff8063cb4e in calltrap ()
    at /usr/src/sys/amd64/amd64/exception.S:218
#8  0xffffffff80ad7a5e in find_block (th=0xffffff8006930000,
    zseg=0xffffff000dafc580, dnp=0xffffff8006d35600, depth=Variable
"depth" is not available.
)
    at
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_traverse.c:400
#9  0xffffffff80ad81a3 in traverse_more (th=0xffffff8006930000)
    at
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_traverse.c:672
#10 0xffffffff80ad8368 in traverse_dsl_dataset (ds=0x23, txg_start=1,
advance=Variable "advance" is not available.

)
    at
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/d---Type 

<return> to continue, or q <return> to quit---
mu_traverse.c:731
#11 0xffffffff80ae41c9 in dsl_dataset_destroy_sync () from
/boot/kernel/zfs.ko
#12 0xffffffff80ae6d2a in dsl_sync_task_group_sync
(dstg=0xffffff000d256600,
    tx=0xffffff000d1fca80)
    at
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_synctask.c:186
#13 0xffffffff80ae6873 in dsl_pool_sync (dp=0xffffff000d521000, txg=3430336)
    at
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_pool.c:316
#14 0xffffffff80af59c3 in spa_sync (spa=0xffffff0006d08000, txg=3430336)
    at
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:3988
#15 0xffffffff80afd772 in txg_sync_thread (arg=Variable "arg" is not
available.
)
    at
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c:352
#16 0xffffffff80385452 in fork_exit (
    callout=0xffffffff80afd490 <txg_sync_thread>, arg=0xffffff000d521000,
    frame=0xffffff80a5b2fc80) at /usr/src/sys/kern/kern_fork.c:811
#17 0xffffffff8063cf2e in fork_trampoline ()
    at /usr/src/sys/amd64/amd64/exception.S:554
#18 0x0000000000000000 in ?? ()
#19 0x0000000000000000 in ?? ()
#20 0x0000000000000001 in ?? ()
#21 0x0000000000000000 in ?? ()
#22 0x0000000000000000 in ?? ()
#23 0x0000000000000000 in ?? ()
#24 0x0000000000000000 in ?? ()
#25 0x0000000000000000 in ?? ()
#26 0x0000000000000000 in ?? ()
#27 0x0000000000000000 in ?? ()
#28 0x0000000000000000 in ?? ()
#29 0x0000000000000000 in ?? ()
#30 0x0000000000000000 in ?? ()
#31 0x0000000000000000 in ?? ()
#32 0x0000000000000000 in ?? ()
#33 0x0000000000000000 in ?? ()
#34 0x0000000000000000 in ?? ()
#35 0x0000000000000000 in ?? ()
#36 0x0000000000000000 in ?? ()
#37 0x0000000000000000 in ?? ()
#38 0x0000000000000000 in ?? ()
#39 0x0000000000000000 in ?? ()
#40 0x0000000000000000 in ?? ()
#41 0x0000000000000000 in ?? ()
#42 0x0000000000dde000 in ?? ()
#43 0xffffffff80950d40 in tdq_cpu ()
#44 0xffffffff8095c940 in tdq_groups ()
#45 0xffffffff8095c8c0 in tdq_cpu ()
#46 0xffffff0009572390 in ?? ()
#47 0x0000000000000000 in ?? ()
#48 0xffffff80a5b2f1e8 in ?? ()
#49 0xffffff0009572390 in ?? ()
#50 0xffffffff803ce154 in sched_switch (td=0xffffffff80afd490, newtd=0x0,
    flags=0) at /usr/src/sys/kern/sched_ule.c:1938
#51 0x0000000000000000 in ?? ()
#52 0x0000000000000000 in ?? ()
#53 0x0000000000000000 in ?? ()
#54 0x0000000000000000 in ?? ()
#55 0x0000000000000000 in ?? ()
#56 0x0000000000000000 in ?? ()
#57 0x0000000000000000 in ?? ()
#58 0x0000000000000000 in ?? ()
#59 0x0000000000000000 in ?? ()
#60 0x0000000000000000 in ?? ()
#61 0x0000000000000000 in ?? ()
#62 0x0000000000000000 in ?? ()
#63 0x0000000000000000 in ?? ()
#64 0x0000000000000000 in ?? ()
#65 0x0000000000000000 in ?? ()
#66 0x0000000000000000 in ?? ()
#67 0x0000000000000000 in ?? ()
#68 0x0000000000000000 in ?? ()
#69 0x0000000000000000 in ?? ()
#70 0x0000000000000000 in ?? ()
#71 0x0000000000000000 in ?? ()
#72 0x0000000000000000 in ?? ()
#73 0x0000000000000000 in ?? ()
#74 0x0000000000000000 in ?? ()
#75 0x0000000000000000 in ?? ()
#76 0x0000000000000000 in ?? ()
#77 0x0000000000000000 in ?? ()
#78 0x0000000000000000 in ?? ()
#79 0x0000000000000000 in ?? ()
#80 0x0000000000000000 in ?? ()
#81 0x0000000000000000 in ?? ()
#82 0x0000000000000000 in ?? ()
#83 0x0000000000000000 in ?? ()
#84 0x0000000000000000 in ?? ()
#85 0x0000000000000000 in ?? ()
#86 0x0000000000000000 in ?? ()
#87 0x0000000000000000 in ?? ()
#88 0x0000000000000000 in ?? ()
#89 0x0000000000000000 in ?? ()
#90 0x0000000000000000 in ?? ()
#91 0x0000000000000000 in ?? ()
#92 0x0000000000000000 in ?? ()
#93 0x0000000000000000 in ?? ()
#94 0x0000000000000000 in ?? ()
#95 0x0000000000000000 in ?? ()
#96 0x0000000000000000 in ?? ()
#97 0x0000000000000000 in ?? ()
#98 0x0000000000000000 in ?? ()
#99 0x0000000000000000 in ?? ()
#100 0x0000000000000000 in ?? ()
#101 0x0000000000000000 in ?? ()
#102 0x0000000000000000 in ?? ()
#103 0x0000000000000000 in ?? ()
#104 0x0000000000000000 in ?? ()
#105 0x0000000000000000 in ?? ()
#106 0x0000000000000000 in ?? ()
#107 0x0000000000000000 in ?? ()
#108 0x0000000000000000 in ?? ()
#109 0x0000000000000000 in ?? ()
#110 0x0000000000000000 in ?? ()
#111 0x0000000000000000 in ?? ()
#112 0x0000000000000000 in ?? ()
#113 0x0000000000000000 in ?? ()
#114 0x0000000000000000 in ?? ()
#115 0x0000000000000000 in ?? ()
#116 0x0000000000000000 in ?? ()
#117 0x0000000000000000 in ?? ()
#118 0x0000000000000000 in ?? ()
Cannot access memory at address 0xffffff80a5b30000
(kgdb) q
# ^D
Script done on Fri Sep 11 17:26:02 2009
Received on Thu Sep 10 2009 - 21:05:55 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:39:55 UTC