Re: ZFS/UMA panic under Parallels

From: Scott Long <scottl_at_samsco.org>
Date: Sun, 22 Apr 2007 04:50:40 -0600
This is actually a problem with the ATA driver, not ZFS.  I think
people are also seeing it under VMWare.  The ATA driver is misusing
semaphores in a way that isn't reliable in the emulated environment.

Scott


Ollivier Robert wrote:
> Hello,
> 
> I'm running CURRENT inside a Parallels VM and I am getting regular panic
> when writing to a ZFS raidz pool.  UMA is complaining and panic().
> 
> It is rather easy to reproduce (cvs update, rsync all generates such
> panics).  Another interesing point is that it always occurs after
> FLUSHCACHE TIMEOUT messages.
> 
> The pool is made of 3 slices ad0s2 ad0s3 from the first virtual disk and
> ad2s1 from a second virtual disk (to get 3 "disks" for raidz).
> 
> Filesystem               1024-blocks    Used    Avail Capacity  Mounted on
> /dev/ad0s1a                  9647558 2708134  6167620    31%    /
> devfs                              1       1        0   100%    /dev
> tank/home                   19787392     128 19787264     0%    /home
> tank                        19787264       0 19787264     0%    /tank
> tank/usr                    19902336  115072 19787264     1%    /tank/usr
> tank/usr/obj                19787264       0 19787264     0%    /tank/usr/obj
> tank/usr/ports              19796224    8960 19787264     0%    /tank/usr/ports
> tank/usr/ports/distfiles    19787264       0 19787264     0%    /tank/usr/ports/distfiles
> tank/usr/src                19983360  196096 19787264     1%    /tank/usr/src
> 
> -----
> GNU gdb 6.1.1 [FreeBSD]
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for details.
> This GDB was configured as "i386-marcel-freebsd".
> 
> Unread portion of the kernel message buffer:
> ad0: FAILURE - FLUSHCACHE timed out
> ad2: FAILURE - FLUSHCACHE timed out
> Slab at 0xc2b90f70, freei 18 = 0.
> panic: Duplicate free of item 0xc2b90948 from zone 0xc1061780(g_bio)
> 
> cpuid = 0
> KDB: enter: panic
> panic: from debugger
> cpuid = 0
> Uptime: 7m28s
> Physical memory: 503 MB
> Dumping 172 MB: 157 141 125 109 93 77 61 45 29 13
> 
> #0  doadump () at pcpu.h:172
> 172		__asm __volatile("movl %%fs:0,%0" : "=r" (td));
> (kgdb) where
> #0  doadump () at pcpu.h:172
> #1  0xc0525a40 in boot (howto=260) at ../../../kern/kern_shutdown.c:409
> #2  0xc0525d4a in panic (fmt=0xc0680204 "from debugger")
>     at ../../../kern/kern_shutdown.c:563
> #3  0xc044a13e in db_panic (addr=-1068219313, have_addr=0, count=-1, 
>     modif=0xdceb09e0 "") at ../../../ddb/db_command.c:433
> #4  0xc044a0d7 in db_command (last_cmdp=0xc0705f64, cmd_table=0x0)
>     at ../../../ddb/db_command.c:401
> #5  0xc044a192 in db_command_loop () at ../../../ddb/db_command.c:453
> #6  0xc044bddd in db_trap (type=3, code=0) at ../../../ddb/db_main.c:222
> #7  0xc0544728 in kdb_trap (type=3, code=0, tf=0xdceb0b78)
>     at ../../../kern/subr_kdb.c:502
> #8  0xc0658871 in trap (frame=0xdceb0b78) at ../../../i386/i386/trap.c:621
> #9  0xc064319b in calltrap () at ../../../i386/i386/exception.s:139
> #10 0xc054444f in kdb_enter (msg=0x12 <Address 0x12 out of bounds>)
>     at cpufunc.h:60
> #11 0xc0525cfc in panic (
>     fmt=0xc06ac621 "Duplicate free of item %p from zone %p(%s)\n")
>     at ../../../kern/kern_shutdown.c:547
> #12 0xc06118f4 in uma_dbg_free (zone=0xc1061780, slab=0xc2b90f70, 
>     item=0xc2b90948) at ../../../vm/uma_dbg.c:302
> #13 0xc06100f3 in uma_zfree_arg (zone=0xc1061780, item=0xc2b90948, udata=0x0)
>     at ../../../vm/uma_core.c:2261
> #14 0xc04f207b in g_destroy_bio (bp=0xc2b90948) at uma.h:305
> #15 0xc04f090a in g_disk_done (bp=0xc2b90948) at ../../../geom/geom_disk.c:209
> #16 0xc05736e0 in biodone (bp=0xc2b90948) at ../../../kern/vfs_bio.c:3006
> #17 0xc045f8c2 in ad_done (request=0xc2b7b480)
>     at ../../../dev/ata/ata-disk.c:304
> #18 0xc0464f50 in ata_completed (context=0xc2b7b480, dummy=0)
>     at ../../../dev/ata/ata-queue.c:479
> #19 0xc054bd0c in taskqueue_run (queue=0xc2a0a080)
>     at ../../../kern/subr_taskqueue.c:255
> #20 0xc054bea2 in taskqueue_swi_run (dummy=0x0)
>     at ../../../kern/subr_taskqueue.c:297
> #21 0xc0510f19 in ithread_execute_handlers (p=0xc299e240, ie=0xc2a0a000)
>     at ../../../kern/kern_intr.c:682
> #22 0xc0511047 in ithread_loop (arg=0xc2a12010)
>     at ../../../kern/kern_intr.c:766
> #23 0xc0510018 in fork_exit (callout=0xc0510fe0 <ithread_loop>, 
>     arg=0xc2a12010, frame=0xdceb0d38) at ../../../kern/kern_fork.c:814
> #24 0xc0643210 in fork_trampoline () at ../../../i386/i386/exception.s:205
> -----
> 
> dmesg
> -----
> Copyright (c) 1992-2007 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
> 	The Regents of the University of California. All rights reserved.
> FreeBSD is a registered trademark of The FreeBSD Foundation.
> FreeBSD 7.0-CURRENT #0: Sun Apr 22 11:05:32 CEST 2007
>     roberto_at_vm1.keltia.net:/usr/osrc/sys/i386/compile/VM7
> Timecounter "i8254" frequency 1193182 Hz quality 0
> CPU: Intel(R) Core(TM)2 CPU         T7600  _at_ 2.33GHz (1524.50-MHz 686-class CPU)
>   Origin = "GenuineIntel"  Id = 0x6f6  Stepping = 6
>   Features=0xf80b9b9<FPU,PSE,TSC,MSR,MCE,CX8,SEP,MTRR,PGE,CMOV,MMX,FXSR,SSE,SSE2,SS>
>   Features2=0x1<SSE3>
> real memory  = 536870912 (512 MB)
> avail memory = 515600384 (491 MB)
> kbd1 at kbdmux0
> cpu0 on motherboard
> pcib0: <Intel 82815 (i815 GMCH) Host To Hub bridge> pcibus 0 on motherboard
> pir0: <PCI Interrupt Routing Table: 9 Entries> on motherboard
> pci0: <PCI bus> on pcib0
> vgapci0: <VGA-compatible display> port 0x4000-0x40ff,0x4400-0x441f mem 0xc0000000-0xc0ffffff at device 2.0 on pci0
> pci0: <bridge> at device 3.0 (no driver attached)
> ed0: <RealTek 8029> port 0x4c00-0x4c1f irq 10 at device 5.0 on pci0
> ed0: [ITHREAD]
> ed0: using obsoleted if_watchdog interface
> ed0: Ethernet address: 00:ee:42:8a:ed:eb
> uhci0: <Intel 82801FB/FR/FW/FRW (ICH6) USB controller USB-A> port 0x5000-0x501f irq 11 at device 29.0 on pci0
> uhci0: [GIANT-LOCKED]
> uhci0: [ITHREAD]
> usb0: <Intel 82801FB/FR/FW/FRW (ICH6) USB controller USB-A> on uhci0
> usb0: USB revision 1.0
> uhub0: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb0
> uhub0: 2 ports with 2 removable, self powered
> ehci0: <Intel 82801FB (ICH6) USB 2.0 controller> mem 0xc1000000-0xc10003ff irq 9 at device 29.7 on pci0
> ehci0: [GIANT-LOCKED]
> ehci0: [ITHREAD]
> usb1: EHCI version 1.0
> usb1: wrong number of companions (0 != 1)
> usb1: <Intel 82801FB (ICH6) USB 2.0 controller> on ehci0
> usb1: USB revision 2.0
> uhub1: <Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1> on usb1
> uhub1: 8 ports with 8 removable, self powered
> agp0: <Intel 82815 (i815 GMCH) host to PCI bridge> on hostb0
> isab0: <PCI-ISA bridge> at device 31.0 on pci0
> isa0: <ISA bus> on isab0
> atapci0: <Intel ICH2 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x6c00-0x6c0f at device 31.1 on pci0
> ata0: <ATA channel 0> on atapci0
> ata0: [ITHREAD]
> ata1: <ATA channel 1> on atapci0
> ata1: [ITHREAD]
> pci0: <multimedia, audio> at device 31.5 (no driver attached)
> pmtimer0 on isa0
> orm0: <ISA Option ROM> at iomem 0xc0000-0xc9fff pnpid ORM0000 on isa0
> sc0: <System console> at flags 0x100 on isa0
> sc0: VGA <16 virtual consoles, flags=0x300>
> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
> atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
> atkbd0: <AT Keyboard> irq 1 on atkbdc0
> kbd0 at atkbd0
> atkbd0: [GIANT-LOCKED]
> atkbd0: [ITHREAD]
> psm0: <PS/2 Mouse> irq 12 on atkbdc0
> psm0: [GIANT-LOCKED]
> psm0: [ITHREAD]
> psm0: model IntelliMouse Explorer, device ID 4
> sio0: configured irq 4 not in bitmap of probed irqs 0
> sio0: port may not be enabled
> sio0: configured irq 4 not in bitmap of probed irqs 0
> sio0: port may not be enabled
> sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
> sio0: type 8250 or not responding
> sio0: [FILTER]
> sio1: configured irq 3 not in bitmap of probed irqs 0
> sio1: port may not be enabled
> Timecounter "TSC" frequency 1524502532 Hz quality 800
> Timecounters tick every 1.000 msec
> ad0: 32000MB <Virtual HDD [0] FWR10003> at ata0-master UDMA100
> acd0: CDROM <PRL CD-ROM [1]/FWR10003> at ata0-slave PIO4
> ad2: 10000MB <Virtual HDD [2] FWR10003> at ata1-master UDMA100
> WARNING: ZFS is considered to be an experimental feature in FreeBSD.
> Trying to mount root from ufs:/dev/ad0s1a
> ZFS filesystem version 6
> ZFS storage pool version 6
> -----
> 
> VM7
> -----
> #
> # VM7 -- Generic kernel configuration file for FreeBSD/i386
> #
> # $FreeBSD: src/sys/i386/conf/GENERIC,v 1.460 2006/12/13 03:41:47 yongari Exp $
> 
> cpu		I686_CPU
> ident		VM7
> 
> 
> makeoptions	DEBUG=-g		# Build kernel with gdb(1) debug symbols
> 
> options 	SCHED_4BSD		# 4BSD scheduler
> options 	PREEMPTION		# Enable kernel thread preemption
> options 	INET			# InterNETworking
> options 	INET6			# IPv6 communications protocols
> options 	FFS			# Berkeley Fast Filesystem
> options 	SOFTUPDATES		# Enable FFS soft updates support
> options 	UFS_ACL			# Support for access control lists
> options 	UFS_DIRHASH		# Improve performance on big directories
> options 	COMPAT_43TTY		# BSD 4.3 TTY compat [KEEP THIS!]
> options 	COMPAT_FREEBSD4		# Compatible with FreeBSD4
> options 	COMPAT_FREEBSD5		# Compatible with FreeBSD5
> options 	COMPAT_FREEBSD6		# Compatible with FreeBSD6
> options 	KTRACE			# ktrace(1) support
> options 	_KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions
> options 	KBD_INSTALL_CDEV	# install a CDEV entry in /dev
> options 	ADAPTIVE_GIANT		# Giant mutex is adaptive.
> options 	STOP_NMI		# Stop CPUS using NMI instead of IPI
> 
> # Debugging for use in -current
> options 	KDB			# Enable kernel debugger support.
> options 	DDB			# Support DDB.
> options 	INVARIANTS		# Enable calls of extra sanity checking
> options 	INVARIANT_SUPPORT	# Extra sanity checks of internal structures, required by INVARIANTS
> 
> # To make an SMP kernel, the next two lines are needed
> options 	SMP			# Symmetric MultiProcessor Kernel
> device		apic			# I/O APIC
> 
> # Bus support.
> device		pci
> 
> # Floppy drives
> device		fdc
> 
> # ATA and ATAPI devices
> device		ata
> device		atadisk		# ATA disk drives
> device		ataraid		# ATA RAID drives
> device		atapicd		# ATAPI CDROM drives
> options 	ATA_STATIC_ID	# Static device numbering
> 
> # SCSI peripherals
> device		scbus		# SCSI bus (required for SCSI)
> device		da		# Direct Access (disks)
> device		cd		# CD
> device		pass		# Passthrough device (direct SCSI access)
> 
> # atkbdc0 controls both the keyboard and the PS/2 mouse
> device		atkbdc		# AT keyboard controller
> device		atkbd		# AT keyboard
> device		psm		# PS/2 mouse
> 
> device		kbdmux		# keyboard multiplexer
> 
> device		vga		# VGA video card driver
> 
> device		splash		# Splash screen and screen saver support
> 
> # syscons is the default console driver, resembling an SCO console
> device		sc
> 
> device		agp		# support several AGP chipsets
> 
> # Power management support (see NOTES for more options)
> #device		apm
> # Add suspend/resume support for the i8254.
> device		pmtimer
> 
> # Serial (COM) ports
> device		sio		# 8250, 16[45]50 based serial ports
> device		uart		# Generic UART driver
> 
> # PCI Ethernet NICs.
> device		de		# DEC/Intel DC21x4x (``Tulip'')
> device		em		# Intel PRO/1000 adapter Gigabit Ethernet Card
> device		ixgb		# Intel PRO/10GbE Ethernet Card
> device		le		# AMD Am7900 LANCE and Am79C9xx PCnet
> device		txp		# 3Com 3cR990 (``Typhoon'')
> device		vx		# 3Com 3c590, 3c595 (``Vortex'')
> 
> # PCI Ethernet NICs that use the common MII bus controller code.
> # NOTE: Be sure to keep the 'device miibus' line in order to use these NICs!
> device		miibus		# MII bus support
> device		ed		# NE[12]000, SMC Ultra, 3c503, DS8390 cards
> 
> device		loop		# Network loopback
> device		random		# Entropy device
> device		ether		# Ethernet support
> device		tun		# Packet tunnel.
> device		pty		# Pseudo-ttys (telnet etc)
> 
> # The `bpf' device enables the Berkeley Packet Filter.
> # Be aware of the administrative consequences of enabling this!
> # Note that 'bpf' is required for DHCP.
> device		bpf		# Berkeley packet filter
> 
> # USB support
> device		uhci		# UHCI PCI->USB interface
> device		ohci		# OHCI PCI->USB interface
> device		ehci		# EHCI PCI->USB interface (USB 2.0)
> device		usb		# USB Bus (required)
> device		ugen		# Generic
> device		ukbd		# Keyboard
> device		umass		# Disks/Mass storage - Requires scbus and da
> device		ums		# Mouse
> 
> # FireWire support
> device		firewire	# FireWire bus code
> device		sbp		# SCSI over FireWire (Requires scbus and da)
> device		fwe		# Ethernet over FireWire (non-standard!)
> -----
Received on Sun Apr 22 2007 - 08:51:08 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:39:09 UTC