Panic / Solaris Assert / ZAP.c:479

From: Larry Rosenman <ler_at_lerctr.org>
Date: Sat, 25 Jan 2014 15:29:55 -0600
Ok, I just did a full rebuild at -CURRENT -HEAD and still get this (basic) panic.

How can I get some help to fix it?

This version stays up:
FreeBSD borg.lerctr.org 11.0-CURRENT FreeBSD 11.0-CURRENT #113 r260864: Fri Jan 17 19:10:06 CST 2014     root_at_:/usr/obj/usr/src/sys/BORG-DTRACE  amd64

Below is the latest panic.
borg.lerctr.org dumped core - see /var/crash/vmcore.2

Sat Jan 25 15:25:30 CST 2014

FreeBSD borg.lerctr.org 11.0-CURRENT FreeBSD 11.0-CURRENT #3 r261166: Sat Jan 25 14:55:24 CST 2014     root_at_borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER  amd64

panic: solaris assert: l->l_phys->l_hdr.lh_pad1 == 0 (0xffff000000000000 == 0x0), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c, line: 479

GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...

Unread portion of the kernel message buffer:
Copyright (c) 1992-2014 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.0-CURRENT #3 r261166: Sat Jan 25 14:55:24 CST 2014
    root_at_borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64
FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610
info: [drm] Initialized drm 1.1.0 20060810
CPU: Intel(R) Xeon(R) CPU           E5410  _at_ 2.33GHz (2327.55-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x10676  Family=0x6  Model=0x17  Stepping=6
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0xce3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1>
  AMD Features=0x20100800<SYSCALL,NX,LM>
  AMD Features2=0x1<LAHF>
  TSC: P-state invariant, performance statistics
real memory  = 68719476736 (65536 MB)
avail memory = 65641644032 (62600 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <PTLTD  	 APIC  >
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 2 package(s) x 4 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
 cpu2 (AP): APIC ID:  2
 cpu3 (AP): APIC ID:  3
 cpu4 (AP): APIC ID:  4
 cpu5 (AP): APIC ID:  5
 cpu6 (AP): APIC ID:  6
 cpu7 (AP): APIC ID:  7
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 24-47 on motherboard
random: <Software, Yarrow> initialized
kbd1 at kbdmux0
cryptosoft0: <software crypto> on motherboard
acpi0: <PTLTD   RSDT> on motherboard
acpi0: Power Button (fixed)
unknown: I/O range not supported
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
cpu4: <ACPI CPU> on acpi0
cpu5: <ACPI CPU> on acpi0
cpu6: <ACPI CPU> on acpi0
cpu7: <ACPI CPU> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff irq 0,8 on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 350
Event timer "HPET1" frequency 14318180 Hz quality 340
Event timer "HPET2" frequency 14318180 Hz quality 340
atrtc0: <AT realtime clock> port 0x70-0x71 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0
pci1: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> irq 16 at device 0.0 on pci1
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> irq 16 at device 0.0 on pci2
pci3: <ACPI PCI bus> on pcib3
pcib4: <ACPI PCI-PCI bridge> at device 0.0 on pci3
pci4: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> at device 0.2 on pci3
pci5: <ACPI PCI bus> on pcib5
pcib6: <ACPI PCI-PCI bridge> irq 18 at device 2.0 on pci2
pci6: <ACPI PCI bus> on pcib6
em0: <Intel(R) PRO/1000 Network Connection 7.3.8> port 0x2000-0x201f mem 0xd9220000-0xd923ffff,0xd9200000-0xd921ffff irq 18 at device 0.0 on pci6
em0: Using an MSI interrupt
em0: Ethernet address: 00:30:48:f2:29:9c
em1: <Intel(R) PRO/1000 Network Connection 7.3.8> port 0x2020-0x203f mem 0xd9260000-0xd927ffff,0xd9240000-0xd925ffff irq 19 at device 0.1 on pci6
em1: Using an MSI interrupt
em1: Ethernet address: 00:30:48:f2:29:9d
pcib7: <ACPI PCI-PCI bridge> at device 0.3 on pci1
pci7: <ACPI PCI bus> on pcib7
pcib8: <ACPI PCI-PCI bridge> at device 4.0 on pci0
pci8: <ACPI PCI bus> on pcib8
vgapci0: <VGA-compatible display> port 0x3000-0x307f mem 0xd8000000-0xd8ffffff,0xc0000000-0xc7ffffff,0xc8000000-0xc9ffffff irq 16 at device 0.0 on pci8
nvidia0: <GeForce GT 610> on vgapci0
vgapci0: child nvidia0 requested pci_enable_io
vgapci0: child nvidia0 requested pci_enable_io
hdac0: <NVIDIA GF119 HDA Controller> mem 0xd9000000-0xd9003fff irq 17 at device 0.1 on pci8
pcib9: <ACPI PCI-PCI bridge> at device 6.0 on pci0
pci9: <ACPI PCI bus> on pcib9
pci0: <base peripheral> at device 8.0 (no driver attached)
pcib10: <ACPI PCI-PCI bridge> irq 17 at device 28.0 on pci0
pci10: <ACPI PCI bus> on pcib10
pcib11: <PCI-PCI bridge> irq 16 at device 0.0 on pci10
pci11: <PCI bus> on pcib11
pcm0: <Envy24HT audio (Generic)> port 0x4080-0x409f,0x4000-0x407f irq 16 at device 0.0 on pci11
pcm0: system configuration
  SubVendorID: 0x1412, SubDeviceID: 0x2403
  XIN2 Clock Source: 24.576MHz(96kHz*256)
  MPU-401 UART(s) #: not implemented
  ADC #: 1 and SPDIF receiver connected
  DAC #: 4
  Multi-track converter type: AC'97(SDATA_OUT:packed)
  S/PDIF(IN/OUT): 1/1 ID# 0x00
  GPIO(mask/dir/state): 0xff/0xff/0xff
uhci0: <Intel 631XESB/632XESB/3100 USB controller USB-1> port 0x1800-0x181f irq 17 at device 29.0 on pci0
usbus0 on uhci0
uhci1: <Intel 631XESB/632XESB/3100 USB controller USB-2> port 0x1820-0x183f irq 19 at device 29.1 on pci0
usbus1 on uhci1
uhci2: <Intel 631XESB/632XESB/3100 USB controller USB-3> port 0x1840-0x185f irq 18 at device 29.2 on pci0
usbus2 on uhci2
ehci0: <Intel 63XXESB USB 2.0 controller> mem 0xd9600400-0xd96007ff irq 17 at device 29.7 on pci0
usbus3: EHCI version 1.0
usbus3 on ehci0
pcib12: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci12: <ACPI PCI bus> on pcib12
vgapci1: <VGA-compatible display> port 0x5000-0x50ff mem 0xd0000000-0xd7ffffff,0xd9300000-0xd930ffff irq 18 at device 1.0 on pci12
drmn1: <ATI ES1000 RN50> on vgapci1
info: [drm] RADEON_IS_PCI
info: [drm] initializing kernel modesetting (RV100 0x1002:0x515E 0x15D9:0x8080).
info: [drm] register mmio base: 0xD9300000
info: [drm] register mmio size: 65536
info: [drm] radeon_atrm_get_bios: ===> Try ATRM...
info: [drm] radeon_atrm_get_bios: pci_find_class() found: 0:8:0:0, vendor=10de, device=104a
info: [drm] radeon_atrm_get_bios: Get ACPI device handle
info: [drm] radeon_acpi_vfct_bios: ===> Try VFCT...
info: [drm] radeon_acpi_vfct_bios: Get "VFCT" ACPI table
info: [drm] radeon_acpi_vfct_bios: Failed to get "VFCT" table: AE_NOT_FOUND
info: [drm] igp_read_bios_from_vram: ===> Try IGP's VRAM...
info: [drm] igp_read_bios_from_vram: VRAM base address: 0xd0000000
info: [drm] igp_read_bios_from_vram: Map address: 0xfffff800d0000000 (262144 bytes)
info: [drm] igp_read_bios_from_vram: Incorrect BIOS signature: 0x0000
info: [drm] radeon_read_bios: ===> Try PCI Expansion ROM...
info: [drm] radeon_read_bios: Map address: 0xfffff800000c0000 (131072 bytes)
drmn1: info: VRAM: 128M 0x00000000D0000000 - 0x00000000D7FFFFFF (16M used)
drmn1: info: GTT: 512M 0x00000000B0000000 - 0x00000000CFFFFFFF
info: [drm] Supports vblank timestamp caching Rev 1 (10.10.2010).
info: [drm] Driver supports precise vblank timestamp query.
info: [drm] radeon: irq initialized.
info: [drm] Detected VRAM RAM=128M, BAR=128M
info: [drm] RAM width 64bits SDR
[TTM] Zone  kernel: Available graphics memory: 33006090 kiB
[TTM] Zone   dma32: Available graphics memory: 2097152 kiB
[TTM] Initializing pool allocator
info: [drm] radeon: 16M of VRAM memory ready
info: [drm] radeon: 512M of GTT memory ready.
info: [drm] GART: num cpu pages 131072, num gpu pages 131072
info: [drm] PCI GART of 512M enabled (table at 0x0000000011200000).
drmn1: info: WB disabled
drmn1: info: fence driver on ring 0 use gpu addr 0x00000000b0000000 and cpu addr 0x0xfffff8001083c000
info: [drm] Loading R100 Microcode
info: [drm] radeon: ring at 0x00000000B0001000
info: [drm] ring test succeeded in 1 usecs
info: [drm] ib test succeeded in 0 usecs
info: [drm] radeon_device_init: Taking over the fictitious range 0xd0000000-0xd4000000
iicbus0: <Philips I2C bus> on iicbb0 addr 0xff
iic0: <I2C generic I/O> on iicbus0
iicbus1: <Philips I2C bus> on iicbb1 addr 0xff
iic1: <I2C generic I/O> on iicbus1
iicbus2: <Philips I2C bus> on iicbb2 addr 0xff
iic2: <I2C generic I/O> on iicbus2
iicbus3: <Philips I2C bus> on iicbb3 addr 0xff
iic3: <I2C generic I/O> on iicbus3
info: [drm] Radeon Display Connectors
info: [drm] Connector 0:
info: [drm]   VGA-1
info: [drm]   DDC: 0x60 0x60 0x60 0x60 0x60 0x60 0x60 0x60
info: [drm]   Encoders:
info: [drm]     CRT1: INTERNAL_DAC1
info: [drm] Connector 1:
info: [drm]   DVI-I-1
info: [drm]   HPD2
info: [drm]   DDC: 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c
info: [drm]   Encoders:
info: [drm]     CRT2: INTERNAL_DAC2
info: [drm]     DFP2: INTERNAL_DVO1
composite sync not supported
composite sync not supported
info: [drm] fb mappable at 0xD0040000
info: [drm] vram apper at 0xD0000000
info: [drm] size 2076672
info: [drm] fb depth is 8
info: [drm]    pitch is 1920
fbd1 on drmn1
vt_allocate: Replace existing VT driver.
info: [drm] Initialized radeon 2.29.0 20080528
vgapci1: Boot video device
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel 63XXESB2 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1860-0x186f at device 31.1 on pci0
ata0: <ATA channel> at channel 0 on atapci0
ahci0: <Intel ESB2 AHCI SATA controller> port 0x18a0-0x18a7,0x1874-0x1877,0x1878-0x187f,0x1870-0x1873,0x1880-0x189f mem 0xd9600800-0xd9600bff irq 19 at device 31.2 on pci0
ahci0: AHCI v1.10 with 6 3Gbps ports, Port Multiplier supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich5: <AHCI channel> at channel 5 on ahci0
ichsmb0: <Intel 631xESB/6321ESB (ESB2) SMBus controller> port 0x1100-0x111f irq 19 at device 31.3 on pci0
smbus0: <System Management Bus> on ichsmb0
acpi_button0: <Power Button> on acpi0
ipmi0: <IPMI System Interface> port 0xca2-0xca3 on acpi0
ipmi0: KCS mode found at io 0xca2 on acpi
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
ppc0: <Parallel port> port 0x378-0x37f,0x778-0x77f irq 7 drq 3 on acpi0
ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode
ppc0: FIFO with 16/16/9 bytes threshold
ppbus0: <Parallel port bus> on ppc0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
ichwd0 on isa0
orm0: <ISA Option ROM> at iomem 0xc0000-0xcafff on isa0
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
coretemp1: <CPU On-Die Thermal Sensors> on cpu1
est1: <Enhanced SpeedStep Frequency Control> on cpu1
p4tcc1: <CPU Frequency Thermal Control> on cpu1
coretemp2: <CPU On-Die Thermal Sensors> on cpu2
est2: <Enhanced SpeedStep Frequency Control> on cpu2
p4tcc2: <CPU Frequency Thermal Control> on cpu2
coretemp3: <CPU On-Die Thermal Sensors> on cpu3
est3: <Enhanced SpeedStep Frequency Control> on cpu3
p4tcc3: <CPU Frequency Thermal Control> on cpu3
coretemp4: <CPU On-Die Thermal Sensors> on cpu4
est4: <Enhanced SpeedStep Frequency Control> on cpu4
p4tcc4: <CPU Frequency Thermal Control> on cpu4
coretemp5: <CPU On-Die Thermal Sensors> on cpu5
est5: <Enhanced SpeedStep Frequency Control> on cpu5
p4tcc5: <CPU Frequency Thermal Control> on cpu5
coretemp6: <CPU On-Die Thermal Sensors> on cpu6
est6: <Enhanced SpeedStep Frequency Control> on cpu6
p4tcc6: <CPU Frequency Thermal Control> on cpu6
coretemp7: <CPU On-Die Thermal Sensors> on cpu7
est7: <Enhanced SpeedStep Frequency Control> on cpu7
p4tcc7: <CPU Frequency Thermal Control> on cpu7
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 1.000 msec
vboxdrv: fAsync=0 offMin=0x387 offMax=0x50f
hdacc0: <NVIDIA (0x001c) HDA CODEC> at cad 0 on hdac0
hdaa0: <NVIDIA (0x001c) Audio Function Group> at nid 1 on hdacc0
pcm1: <NVIDIA (0x001c) (HDMI/DP 8ch)> at nid 4 on hdaa0
pcm2: <NVIDIA (0x001c) (HDMI/DP 8ch)> at nid 5 on hdaa0
random: unblocking device.
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 480Mbps High Speed USB v2.0
ugen3.1: <Intel> at usbus3
uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3
ugen2.1: <Intel> at usbus2
uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
ugen1.1: <Intel> at usbus1
uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
ugen0.1: <Intel> at usbus0
uhub3: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ipmi0: IPMI device rev. 1, firmware rev. 1.64, version 2.0
ata0: DMA limited to UDMA33, controller found non-ATA66 cable
ipmi0: Number of channels 8
ipmi0: Attached watchdog
uhub2: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub3: 2 ports with 2 removable, self powered
ada0 at ahcich0 bus 0 scbus1 target 0 lun 0
ada0: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
ada0: Serial Number 5YDA1ZL4
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada0: quirks=0x1<4K>
ada0: Previously was known as ad4
ada1 at ahcich1 bus 0 scbus2 target 0 lun 0
ada1: <ST2000DL003-9VT166 CC3D> ATA-8 SATA 3.x device
ada1: Serial Number 5YD6FPLG
ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada1: quirks=0x1<4K>
ada1: Previously was known as ad6
ada2 at ahcich2 bus 0 scbus3 target 0 lun 0
ada2: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
ada2: Serial Number 5YDA3PC5
ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada2: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada2: quirks=0x1<4K>
ada2: Previously was known as ad8
ada3 at ahcich3 bus 0 scbus4 target 0 lun 0
ada3: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
ada3: Serial Number 5YD9Y0P4
ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada3: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada3: quirks=0x1<4K>
ada3: Previously was known as ad10
ada4 at ahcich4 bus 0 scbus5 target 0 lun 0
ada4: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
ada4: Serial Number 5YDA1R5W
ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada4: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada4: quirks=0x1<4K>
ada4: Previously was known as ad12
ada5 at ahcich5 bus 0 scbus6 target 0 lun 0
ada5: <ST2000DL003-9VT166 CC3D> ATA-8 SATA 3.x device
ada5: Serial Number 5YD5RBS8
ada5: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada5: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada5: quirks=0x1<4K>
ada5: Previously was known as ad14
cd0 at ata0 bus 0 scbus0 target 0 lun 0
cd0: <Memorex DVD+-RAM 510L v1 MWS7> Removable CD-ROM SCSI-0 device 
cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present
Netvsc initializing... done!
SMP: AP CPU #7 Launched!
SMP: AP CPU #1 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #6 Launched!
SMP: AP CPU #4 Launched!
SMP: AP CPU #5 Launched!
Root mount waiting for: usbus3
uhub0: 6 ports with 6 removable, self powered
Root mount waiting for: usbus3
Root mount waiting for: usbus3
ugen3.2: <Peppercon AG> at usbus3
ukbd0: <Peppercon AG Multidevice, class 0/0, rev 2.00/0.01, addr 2> on usbus3
kbd2 at ukbd0
Trying to mount root from zfs:zroot/ROOT/default []...
ugen0.2: <American Power Conversion> at usbus0
ugen1.2: <Prolific Technology Inc.> at usbus1
<118>Setting hostuuid: 53d19f64-d663-a017-8922-0030488e9ff3.
<118>Setting hostid: 0xf53a926e.
<118>Entropy harvesting: interrupts ethernet point_to_point swi.
<118>Starting file system checks:
<118>Mounting local file systems:.
panic: solaris assert: l->l_phys->l_hdr.lh_pad1 == 0 (0xffff000000000000 == 0x0), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c, line: 479
cpuid = 7
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe100c55d350
kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe100c55d400
vpanic() at vpanic+0x126/frame 0xfffffe100c55d440
panic() at panic+0x43/frame 0xfffffe100c55d4a0
assfail3() at assfail3+0x2f/frame 0xfffffe100c55d4c0
zap_get_leaf_byblk() at zap_get_leaf_byblk+0x2ea/frame 0xfffffe100c55d510
zap_deref_leaf() at zap_deref_leaf+0xd1/frame 0xfffffe100c55d560
fzap_length() at fzap_length+0x31/frame 0xfffffe100c55d5d0
zap_length_uint64() at zap_length_uint64+0xbf/frame 0xfffffe100c55d630
ddt_zap_lookup() at ddt_zap_lookup+0x3d/frame 0xfffffe100c55d790
ddt_lookup() at ddt_lookup+0x17e/frame 0xfffffe100c55d960
zio_ddt_write() at zio_ddt_write+0xf2/frame 0xfffffe100c55dad0
zio_execute() at zio_execute+0x1eb/frame 0xfffffe100c55db30
taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe100c55db80
taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe100c55dbb0
fork_exit() at fork_exit+0x84/frame 0xfffffe100c55dbf0
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe100c55dbf0
--- trap 0, rip = 0, rsp = 0xfffffe100c55dcb0, rbp = 0 ---
KDB: enter: panic
Uptime: 23s
Dumping 2330 out of 64465 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%

Reading symbols from /boot/kernel/linux.ko.symbols...done.
Loaded symbols for /boot/kernel/linux.ko.symbols
Reading symbols from /boot/kernel/if_lagg.ko.symbols...done.
Loaded symbols for /boot/kernel/if_lagg.ko.symbols
Reading symbols from /boot/kernel/snd_envy24ht.ko.symbols...done.
Loaded symbols for /boot/kernel/snd_envy24ht.ko.symbols
Reading symbols from /boot/kernel/snd_spicds.ko.symbols...done.
Loaded symbols for /boot/kernel/snd_spicds.ko.symbols
Reading symbols from /boot/kernel/coretemp.ko.symbols...done.
Loaded symbols for /boot/kernel/coretemp.ko.symbols
Reading symbols from /boot/kernel/ichsmb.ko.symbols...done.
Loaded symbols for /boot/kernel/ichsmb.ko.symbols
Reading symbols from /boot/kernel/smbus.ko.symbols...done.
Loaded symbols for /boot/kernel/smbus.ko.symbols
Reading symbols from /boot/kernel/ichwd.ko.symbols...done.
Loaded symbols for /boot/kernel/ichwd.ko.symbols
Reading symbols from /boot/kernel/cpuctl.ko.symbols...done.
Loaded symbols for /boot/kernel/cpuctl.ko.symbols
Reading symbols from /boot/kernel/crypto.ko.symbols...done.
Loaded symbols for /boot/kernel/crypto.ko.symbols
Reading symbols from /boot/kernel/cryptodev.ko.symbols...done.
Loaded symbols for /boot/kernel/cryptodev.ko.symbols
Reading symbols from /boot/kernel/dtraceall.ko.symbols...done.
Loaded symbols for /boot/kernel/dtraceall.ko.symbols
Reading symbols from /boot/kernel/profile.ko.symbols...done.
Loaded symbols for /boot/kernel/profile.ko.symbols
Reading symbols from /boot/kernel/cyclic.ko.symbols...done.
Loaded symbols for /boot/kernel/cyclic.ko.symbols
Reading symbols from /boot/kernel/dtrace.ko.symbols...done.
Loaded symbols for /boot/kernel/dtrace.ko.symbols
Reading symbols from /boot/kernel/systrace_freebsd32.ko.symbols...done.
Loaded symbols for /boot/kernel/systrace_freebsd32.ko.symbols
Reading symbols from /boot/kernel/systrace.ko.symbols...done.
Loaded symbols for /boot/kernel/systrace.ko.symbols
Reading symbols from /boot/kernel/sdt.ko.symbols...done.
Loaded symbols for /boot/kernel/sdt.ko.symbols
Reading symbols from /boot/kernel/lockstat.ko.symbols...done.
Loaded symbols for /boot/kernel/lockstat.ko.symbols
Reading symbols from /boot/kernel/fasttrap.ko.symbols...done.
Loaded symbols for /boot/kernel/fasttrap.ko.symbols
Reading symbols from /boot/kernel/fbt.ko.symbols...done.
Loaded symbols for /boot/kernel/fbt.ko.symbols
Reading symbols from /boot/kernel/dtnfscl.ko.symbols...done.
Loaded symbols for /boot/kernel/dtnfscl.ko.symbols
Reading symbols from /boot/kernel/dtmalloc.ko.symbols...done.
Loaded symbols for /boot/kernel/dtmalloc.ko.symbols
Reading symbols from /boot/modules/vboxdrv.ko...done.
Loaded symbols for /boot/modules/vboxdrv.ko
Reading symbols from /boot/modules/nvidia.ko...done.
Loaded symbols for /boot/modules/nvidia.ko
Reading symbols from /boot/kernel/ipmi.ko.symbols...done.
Loaded symbols for /boot/kernel/ipmi.ko.symbols
Reading symbols from /boot/kernel/ipmi_linux.ko.symbols...done.
Loaded symbols for /boot/kernel/ipmi_linux.ko.symbols
Reading symbols from /boot/kernel/radeonkms.ko.symbols...done.
Loaded symbols for /boot/kernel/radeonkms.ko.symbols
Reading symbols from /boot/kernel/iicbb.ko.symbols...done.
Loaded symbols for /boot/kernel/iicbb.ko.symbols
Reading symbols from /boot/kernel/iicbus.ko.symbols...done.
Loaded symbols for /boot/kernel/iicbus.ko.symbols
Reading symbols from /boot/kernel/iic.ko.symbols...done.
Loaded symbols for /boot/kernel/iic.ko.symbols
Reading symbols from /boot/kernel/drm2.ko.symbols...done.
Loaded symbols for /boot/kernel/drm2.ko.symbols
Reading symbols from /boot/kernel/radeonkmsfw_R100_cp.ko.symbols...done.
Loaded symbols for /boot/kernel/radeonkmsfw_R100_cp.ko.symbols
Reading symbols from /boot/kernel/fdescfs.ko.symbols...done.
Loaded symbols for /boot/kernel/fdescfs.ko.symbols
Reading symbols from /boot/kernel/linprocfs.ko.symbols...done.
Loaded symbols for /boot/kernel/linprocfs.ko.symbols
#0  doadump (textdump=1) at pcpu.h:219
219	pcpu.h: No such file or directory.
	in pcpu.h
(kgdb) #0  doadump (textdump=1) at pcpu.h:219
#1  0xffffffff809b9c67 in kern_reboot (howto=260)
    at /usr/src/sys/kern/kern_shutdown.c:452
#2  0xffffffff809ba175 in vpanic (fmt=<value optimized out>, 
    ap=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:759
#3  0xffffffff809ba1c3 in panic (fmt=<value optimized out>)
    at /usr/src/sys/kern/kern_shutdown.c:688
#4  0xffffffff80327abf in assfail3 (a=<value optimized out>, 
    lv=<value optimized out>, op=<value optimized out>, 
    rv=<value optimized out>, f=<value optimized out>, 
    l=<value optimized out>)
    at /usr/src/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c:91
#5  0xffffffff803bac1a in zap_get_leaf_byblk (zap=<value optimized out>, 
    blkid=4858, tx=0x0, lt=RW_READER, lp=0xfffffe100c55d5a0)
    at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c:479
#6  0xffffffff803b84f1 in zap_deref_leaf (zap=0xfffff800220bf680, 
    h=1610088556361285632, tx=0x0, lt=RW_READER, lp=0xfffffe100c55d5a0)
    at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c:585
#7  0xffffffff803b9481 in fzap_length (zn=0xfffff80028e95600, 
    integer_size=0xfffffe100c55d658, num_integers=0xfffffe100c55d650)
    at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c:897
#8  0xffffffff803bf5bf in zap_length_uint64 (os=<value optimized out>, 
    zapobj=<value optimized out>, key=<value optimized out>, 
    key_numints=<value optimized out>, integer_size=0xfffffe100c55d658, 
    num_integers=0xfffffe100c55d650)
    at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_micro.c:934
#9  0xffffffff80352e7d in ddt_zap_lookup (os=0xfffff80020793c00, object=155, 
    dde=0xfffff80028e96600)
    at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/ddt_zap.c:64
#10 0xffffffff8035146e in ddt_lookup (ddt=0xfffffe000d515000, 
    bp=<value optimized out>, add=<value optimized out>)
    at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/ddt.c:178
#11 0xffffffff803fb242 in zio_ddt_write (zio=0xfffff800288a0730)
    at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:2242
#12 0xffffffff803f893b in zio_execute (zio=0xfffff800288a0730)
    at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1355
#13 0xffffffff80a01970 in taskqueue_run_locked (queue=0xfffff80020127000)
    at /usr/src/sys/kern/subr_taskqueue.c:355
#14 0xffffffff80a022cb in taskqueue_thread_loop (arg=<value optimized out>)
    at /usr/src/sys/kern/subr_taskqueue.c:576
#15 0xffffffff80989324 in fork_exit (
    callout=0xffffffff80a02230 <taskqueue_thread_loop>, 
    arg=0xfffff800200fd4e0, frame=0xfffffe100c55dc00)
    at /usr/src/sys/kern/kern_fork.c:977
#16 0xffffffff80d95b0e in fork_trampoline ()
    at /usr/src/sys/amd64/amd64/exception.S:605
#17 0x0000000000000000 in ?? ()
Current language:  auto; currently minimal
(kgdb) 

------------------------------------------------------------------------
ps -axlww

UID PID PPID CPU PRI NI   VSZ RSS MWCHAN   STAT TT     TIME COMMAND
  0   0    0   0  -8  0     0   0 -        DLs   -  0:00.10 [kernel]
  0   1    0   0  24  0  9368   0 wait     DLs   -  0:00.00 [init]
  0   2    0   0 -16  0     0   0 ftcl     DL    -  0:00.00 [ftcleanup]
  0   3    0   0 -16  0     0   0 crypto_w DL    -  0:00.00 [crypto]
  0   4    0   0 -16  0     0   0 crypto_r DL    -  0:00.00 [crypto returns]
  0   5    0   0 -16  0     0   0 ccb_scan DL    -  0:00.00 [cam]
  0   6    0   0 -16  0     0   0 -        DL    -  0:00.00 [fdc0]
  0   7    0   0  -8  0     0   0 zio->io_ DL    -  0:00.00 [zfskern]
  0   8    0   0 -16  0     0   0 waiting_ DL    -  0:00.00 [sctp_iterator]
  0   9    0   0 -16  0     0   0 ipmireq  DL    -  0:00.00 [ipmi0: kcs]
  0  10    0   0 -16  0     0   0 audit_wo DL    -  0:00.00 [audit]
  0  11    0   0 155  0     0   0 -        RL    -  0:00.00 [idle]
  0  12    0   0 -84  0     0   0 -        WL    -  0:00.00 [intr]
  0  13    0   0  -8  0     0   0 -        DL    -  0:00.00 [geom]
  0  14    0   0 -16  0     0   0 -        DL    -  0:00.00 [rand_harvestq]
  0  15    0   0 -68  0     0   0 -        DL    -  0:00.00 [usb]
  0  16    0   0 -20  0     0   0 VBoxIS   DL    -  0:00.00 [TIMER]
  0  17    0   0 -16  0     0   0 psleep   DL    -  0:00.00 [pagedaemon]
  0  18    0   0 -16  0     0   0 psleep   DL    -  0:00.00 [vmdaemon]
  0  19    0   0 155  0     0   0 pgzero   DL    -  0:00.00 [pagezero]
  0  20    0   0 -16  0     0   0 psleep   DL    -  0:00.00 [bufdaemon]
  0  21    0   0  16  0     0   0 syncer   DL    -  0:00.00 [syncer]
  0  22    0   0 -16  0     0   0 vlruwt   DL    -  0:00.00 [vnlru]
  0  23    0   0 -16  0     0   0 sdflush  DL    -  0:00.00 [softdepflush]
  0  24    1   0  52  0 16928   0 wait     Ds+   -  0:00.01 [sh]
  0  72   24   0  52  0 16928   0 wait     D+    -  0:00.00 [sh]
  0  75   72   0  52  0 40000   0 tx->tx_s D+    -  0:00.00 [zfs]

------------------------------------------------------------------------
vmstat -s

   158237 cpu context switches
    22164 device interrupts
      800 software interrupts
     5019 traps
     5843 system calls
       23 kernel threads created
       37  fork() calls
       15 vfork() calls
        0 rfork() calls
        0 swap pager pageins
        0 swap pager pages paged in
        0 swap pager pageouts
        0 swap pager pages paged out
      136 vnode pager pageins
      911 vnode pager pages paged in
        0 vnode pager pageouts
        0 vnode pager pages paged out
        0 page daemon wakeups
        0 pages examined by the page daemon
        0 pages reactivated
     1771 copy-on-write faults
       15 copy-on-write optimized faults
     2232 zero fill pages zeroed
        0 zero fill pages prezeroed
        0 intransit blocking page faults
     4649 total VM faults taken
      113 page faults requiring I/O
        0 pages affected by kernel thread creation
     1334 pages affected by  fork()
      525 pages affected by vfork()
        0 pages affected by rfork()
        0 pages cached
    55169 pages freed
        0 pages freed by daemon
        0 pages freed by exiting processes
      465 pages active
      860 pages inactive
        0 pages in VM cache
   190921 pages wired down
 15883166 pages free
     4096 bytes per page
     1886 total name lookups
          cache hits (80% pos + 4% neg) system 0% per-directory
          deletions 0%, falsehits 0%, toolong 0%

------------------------------------------------------------------------
vmstat -m

         Type InUse MemUse HighUse Requests  Size(s)
         cdev     8     2K       -        8  256
     ppbusdev     2     1K       -        2  256
     filedesc    27    54K       -       76  2048
      kdtrace   323    76K       -      515  64,256
         kenv    86    11K       -      114  16,32,64,128
    proc-args     3     1K       -       31  32,64,128
        hhook     2     1K       -        2  256
      ithread   125    22K       -      125  32,128,256
       KTRACE   100    13K       -      100  128
      entropy  1026    65K       -     1026  32,64
     acpiintr     1     1K       -        1  64
       linker   403   101K       -      533  16,32,64,128,256,512,1024,2048,4096
        lockf     2     1K       -        2  64,128
   loginclass     2     1K       -        2  64
        cache     1     1K       -        1  32
       devbuf 17314 34391K       -    18687  16,32,64,128,256,512,1024,2048,4096
         temp    57    18K       -      538  16,32,64,128,256,512,1024,2048
       ip6ndp     3     1K       -        3  64
      ata_pci     1     1K       -        1  64
       acpica  1824   187K       -    63680  16,32,64,128,256,512,1024,2048
       kbdmux     7    18K       -        7  16,512,1024,2048
       module   521    66K       -      521  128
     mtx_pool     2    16K       -        2  
          LED     4     1K       -        4  16,128
      CAM XPT    58     4K       -      308  16,32,64,128,512,1024,2048
          osd     5     1K       -       11  16,32,64,128
     pmchooks     1     1K       -        1  128
      CAM DEV    15    30K       -       24  2048
     acpitask     1     8K       -        1  
         hdaa     4     5K       -        4  512,1024,2048
         pgrp     2     1K       -        2  128
      session     2     1K       -        2  128
         proc     2   256K       -        2  
      subproc   100   145K       -      149  512,4096
         cred    40     7K       -      240  64,256
       plimit     2     1K       -        2  256
      uidinfo     2    33K       -        2  128
         hdac     1     1K       -        1  512
        hdacc     1     1K       -        1  32
        vtbuf    24  5712K       -       24  4096
      acpisem    22     3K       -       22  128
           vt    11     3K       -       11  256
       sysctl     0     0K       -       43  16,32,64
    sysctloid  7607   378K       -     7779  16,32,64,128
    sysctltmp     0     0K       -        6  32,64
      tidhash     1   256K       -        1  
      callout     9  3208K       -        9  
         umtx   666    84K       -      666  128
     p1003.1b     1     1K       -        1  16
         SWAP    12 19681K       -       12  64
          bus  2257   362K       -     8440  16,32,64,128,256,512,1024
       bus-sc   157   310K       -     5479  16,32,64,128,256,512,1024,2048,4096
      devstat    32    65K       -       32  32,4096
 eventhandler   104     9K       -      104  64,128
       DEVFS3   230    58K       -      270  256
         kobj   349  1396K       -     1111  4096
       DEVFS1   202   101K       -      238  512
      Per-cpu     1     1K       -        1  32
        DEVFS    41     1K       -       42  16,128
         rman   360    41K       -      754  16,32,128
         sbuf     0     0K       -     2295  16,32,64,128,256
       sglist     3     1K       -        3  32
    taskqueue   115    17K       -      159  16,32,64,128,256
     terminal    11     3K       -       11  256
       Unitno    28     2K       -      120  32,64
         vmem     2   160K       -        3  
     ioctlops     0     0K       -      158  16,32,64
          iov     1     1K       -       27  64,256,512
          msg     4    30K       -        4  2048,4096
          sem     4   106K       -        4  2048,4096
          shm     1    20K       -        1  
          tty    14    14K       -       14  1024
        shmfd     1     8K       -        1  
          pcb    12  8341K       -       12  16,128,1024,2048
     vfscache     1 16384K       -        1  
     vfs_hash     1  8192K       -        1  
       vnodes     1     1K       -        1  256
        mount   194     7K       -      402  16,32,64,128,256
  vnodemarker     0     0K       -        8  512
          BPF     3     1K       -        3  128
        ifnet     4     7K       -        4  128,2048
       ifaddr    34    10K       -       34  32,64,256,512,2048,4096
        clone     8     1K       -        8  128
       arpcom     2     1K       -        2  16
      lltable     6     3K       -        6  512
     routetbl     5     2K       -        5  256,512
         igmp     3     1K       -        3  256
     sctp_vrf     1     1K       -        1  64
    hostcache     1    28K       -        1  
     syncache     1    64K       -        1  
          mld     3     1K       -        3  128
          rpc     2     1K       -        2  512
audit_evclass   187     6K       -      228  32
    ufs_quota     1  8192K       -        1  
    vm_pgdata     7  8193K       -        7  128
      UMAHash    19    10K       -       19  512
    pfs_nodes    73    19K       -       73  256
  pfs_vncache     2     1K       -        2  64
      scsi_cd     0     0K       -       11  16
         GEOM   404    67K       -     3278  16,32,64,128,256,512,1024,2048
      memdesc     1     4K       -        1  4096
       feeder    20     2K       -       24  32,128
      acpidev    36     3K       -       36  64
     atkbddev     2     1K       -        2  64
      CAM CCB     1     2K       -    22943  2048
        mixer     3    12K       -        3  4096
    raid_data     0     0K       -      480  32,128,256
     CAM path    22     1K       -      110  32
      solaris 326492 166906K       -   672607  16,32,64,128,256,512,1024,2048,4096
         UART     6     5K       -        6  16,1024
   kstat_data     6     1K       -        6  64
   CAM periph    16     4K       -       40  16,32,64,128,256
md_nvidia_data     0     0K       -       78  512
     pci_link    16     2K       -       16  32,64,128
  md_sii_data     0     0K       -       78  512
    acpi_perf     8     1K       -        8  64
    CAM queue    23     8K       -       57  16,32,512
       apmdev     1     1K       -        1  128
   madt_table     0     0K       -        1  4096
CAM dev queue     8     1K       -        8  64
      CAM SIM     8     2K       -        8  256
  ddb_capture     1    48K       -        1  
      io_apic     2     4K       -        2  2048
          MCA     8     1K       -        8  128
          msi     4     1K       -        4  128
     nexusdev     4     1K       -        4  16
       isadev     5     1K       -        5  128
          USB    46    59K       -       52  16,32,64,128,256,512,1024,2048,4096
       USBdev    35     4K       -       35  32,64,128,256
        linux    17     2K       -       17  64
       spicds     7     1K       -        7  128
     envy24ht    15   195K       -       15  64,2048
       cpuctl     1     1K       -        1  64
       crypto     1     1K       -        1  512
       cyclic    32     3K       -       32  16,64,128
          SDT    32     2K       -       32  16,64
          fbt 60661  7839K       -    60661  128
     iprtheap    17    54K       -       17  32,64,128,256
       nvidia   194   864K       -      195  16,32,64,128,256,512,1024,2048,4096
         ipmi     0     0K       -       11  128
     gem_name    59    10K       -       62  32,4096
   drm_global     2     1K       -        2  128,256
   drm_vblank     7     1K       -        7  16,64
      drm_dma     2     1K       -        2  32
    drm_sarea     1     1K       -        1  16
   drm_driver    32  2203K       -       36  16,32,64,128,256,512,1024,4096
drm_ctxbitmap     1     4K       -        1  4096
     drm_sman    13     2K       -       13  128
  drm_hashtab     1  4096K       -        1  
      drm_kms    90    18K       -      108  16,32,64,128,256,2048
       ttm_pd     6     9K       -        6  16,2048
     ttm_rman     2     1K       -        2  256
     ttm_zone     2     1K       -        2  64
  ttm_poolmgr     1     1K       -        1  512
  fdesc_mount     1     1K       -        1  16

------------------------------------------------------------------------
vmstat -z

ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP

UMA Kegs:               384,      0,     267,       3,     267,   0,   0
UMA Zones:             1664,      0,     267,       1,     267,   0,   0
UMA Slabs:              112,      0,   12325,      30,   14833,   0,   0
UMA RCntSlabs:          120,      0,       0,       0,       0,   0,   0
UMA Hash:               256,      0,      58,      32,      77,   0,   0
4 Bucket:                32,      0,    1296,    1184,    9186,   0,   0
6 Bucket:                48,      0,     310,    1018,    1646,   0,   0
8 Bucket:                64,      0,     103,    1199,    2674,   0,   0
12 Bucket:               96,      0,      64,    1084,    1327,   0,   0
16 Bucket:              128,      0,     100,    1171,    1676,  17,   0
32 Bucket:              256,      0,     240,     630,    4097,  49,   0
64 Bucket:              512,      0,     238,     287,    1922,  49,   0
128 Bucket:            1024,      0,    1190,     102,    6750,   0,   0
vmem btag:               56,      0,   14856,     409,   14954, 215,   0
VM OBJECT:              256,      0,     583,     407,    1328,   0,   0
RADIX NODE:             144,      0,    8773,     272,   13246,   0,   0
MAP:                    240,      0,       3,      61,       3,   0,   0
KMAP ENTRY:             128,      0,       8,     271,       8,   0,   0
MAP ENTRY:              128,      0,     103,    1044,    1717,   0,   0
VMSPACE:                448,      0,       4,     172,      54,   0,   0
fakepg:                 104,      0,       0,       0,       0,   0,   0
mt_zone:               4112,      0,     405,       0,     405,   0,   0
16:                      16,      0,       6,     243,       6,   0,   0
16:                      16,      0,    2762,     724,    3015,   0,   0
16:                      16,      0,  154129,     749,  173890,   0,   0
16:                      16,      0,      54,     693,   28214,   0,   0
16:                      16,      0,      62,     436,      63,   0,   0
16:                      16,      0,     690,     804,    5386,   0,   0
16:                      16,      0,      30,     219,      30,   0,   0
16:                      16,      0,       1,     746,      18,   0,   0
32:                      32,      0,      30,     962,     116,   0,   0
32:                      32,      0,    3092,     876,    3286,   0,   0
32:                      32,      0,   74071,     577,  140388,   0,   0
32:                      32,      0,      66,     430,    4216,   0,   0
32:                      32,      0,      18,    1098,      98,   0,   0
32:                      32,      0,     833,     903,    2868,   0,   0
32:                      32,      0,      94,     526,     186,   0,   0
32:                      32,      0,       8,     612,     164,   0,   0
64:                      64,      0,       5,     181,       6,   0,   0
64:                      64,      0,     516,     786,     586,   0,   0
64:                      64,      0,   20714,    1172,   73124,   0,   0
64:                      64,      0,     767,     659,    1637,   0,   0
64:                      64,      0,      80,     106,      80,   0,   0
64:                      64,      0,    9171,     749,    9688,   0,   0
64:                      64,      0,     131,     303,     131,   0,   0
64:                      64,      0,    1038,     884,    1071,   0,   0
128:                    128,      0,      34,     865,      95,   0,   0
128:                    128,      0,    1899,     364,    1908,   0,   0
128:                    128,      0,  121894,   13359,  190707,   0,   0
128:                    128,      0,    1077,     318,   27008,   0,   0
128:                    128,      0,     112,     911,     192,   0,   0
128:                    128,      0,    1877,     882,    4086,   0,   0
128:                    128,      0,      29,     126,      30,   0,   0
128:                    128,      0,     524,     499,     526,   0,   0
256:                    256,      0,       2,      73,       2,   0,   0
256:                    256,      0,     301,     254,     444,   0,   0
256:                    256,      0,    7514,    4696,   54916,   0,   0
256:                    256,      0,      64,     131,     740,   0,   0
256:                    256,      0,      28,     287,     348,   0,   0
256:                    256,      0,     903,     447,    1957,   0,   0
256:                    256,      0,      31,      44,      31,   0,   0
256:                    256,      0,       4,     251,      18,   0,   0
512:                    512,      0,       2,      33,       9,   0,   0
512:                    512,      0,      88,     206,     244,   0,   0
512:                    512,      0,     347,     500,   78882,   0,   0
512:                    512,      0,      16,      47,      35,   0,   0
512:                    512,      0,       3,      32,       3,   0,   0
512:                    512,      0,     327,     198,    2807,   0,   0
512:                    512,      0,      47,      16,      47,   0,   0
512:                    512,      0,       1,      62,       2,   0,   0
1024:                  1024,      0,       0,      92,      99,   0,   0
1024:                  1024,      0,       9,      19,       9,   0,   0
1024:                  1024,      0,    8351,     125,   23312,   0,   0
1024:                  1024,      0,       5,      23,    2074,   0,   0
1024:                  1024,      0,      16,      12,      16,   0,   0
1024:                  1024,      0,     230,      82,    1273,   0,   0
1024:                  1024,      0,       2,      14,       2,   0,   0
1024:                  1024,      0,       0,       0,       0,   0,   0
2048:                  2048,      0,       2,       4,       9,   0,   0
2048:                  2048,      0,       6,       4,       6,   0,   0
2048:                  2048,      0,      11,      25,      39,   0,   0
2048:                  2048,      0,       8,      58,   22955,   0,   0
2048:                  2048,      0,      21,       9,      30,   0,   0
2048:                  2048,      0,      48,      38,     466,   0,   0
2048:                  2048,      0,       5,       1,       5,   0,   0
2048:                  2048,      0,       0,       0,       0,   0,   0
4096:                  4096,      0,       0,       0,       0,   0,   0
4096:                  4096,      0,       5,       0,       5,   0,   0
4096:                  4096,      0,     298,       2,     497,   0,   0
4096:                  4096,      0,      10,       0,      12,   0,   0
4096:                  4096,      0,      16,       0,      17,   0,   0
4096:                  4096,      0,      25,       0,     415,   0,   0
4096:                  4096,      0,      15,       0,      15,   0,   0
4096:                  4096,      0,     349,       9,    1111,   0,   0
uint64 pcpu:              8,      0,    1398,     138,    1398,   0,   0
SLEEPQUEUE:              88,      0,     334,     565,     334,   0,   0
Files:                   80,      0,       8,    1021,     395,   0,   0
rl_entry:                40,      0,       6,     588,       6,   0,   0
TURNSTILE:              136,      0,     334,     226,     334,   0,   0
umtx pi:                 96,      0,       0,       0,       0,   0,   0
MAC labels:              40,      0,       0,       0,       0,   0,   0
PROC:                  1208,      0,      26,      46,      75,   0,   0
THREAD:                1168,      0,     295,      38,     438,   0,   0
cpuset:                  72,      0,     282,     488,     425,   0,   0
cyclic_id_cache:         64,      0,       0,       0,       0,   0,   0
audit_record:          1248,      0,       0,       0,       0,   0,   0
mbuf_packet:            256, 25720710,       0,      10,       0,   0,   0
mbuf:                   256, 25720710,       1,     134,       1,   0,   0
mbuf_cluster:          2048, 4018860,       0,       0,       0,   0,   0
mbuf_jumbo_page:       4096, 2009429,       0,       0,       0,   0,   0
mbuf_jumbo_9k:         9216, 1786158,       0,       0,       0,   0,   0
mbuf_jumbo_16k:       16384, 1339616,       0,       0,       0,   0,   0
mbuf_ext_refcnt:          4,      0,       0,       0,       0,   0,   0
sendfile_sync:          128,      0,       0,       0,       0,   0,   0
dtrace_state_cache:    4096,      0,       0,       0,       0,   0,   0
g_bio:                  248,      0,       3,     669,   68266,   0,   0
DMAR_MAP_ENTRY:         120,      0,       0,       0,       0,   0,   0
ttyinq:                 160,      0,      15,      57,      15,   0,   0
ttyoutq:                256,      0,       8,      67,       8,   0,   0
ata_request:            336,      0,       0,      88,      42,   0,   0
vtnet_tx_hdr:            24,      0,       0,       0,       0,   0,   0
cryptop:                 88,      0,       0,       0,       0,   0,   0
cryptodesc:              72,      0,       0,       0,       0,   0,   0
nv_stack_t:           12288,      0,       2,       1,       6,   0,   0
FPU_save_area:          512,      0,       0,       0,       0,   0,   0
taskq_zone:              48,      0,       0,     664,      47,   0,   0
taskq_zone:              48,      0,       0,       0,       0,   0,   0
VNODE:                  472,      0,    4245,     139,    4246,   0,   0
VNODEPOLL:              112,      0,       0,       0,       0,   0,   0
BUF TRIE:               144,      0,       0,  105948,       0,   0,   0
NAMEI:                 1024,      0,       0,     104,     887,   0,   0
S VFS Cache:            108,      0,     247,     908,     285,   0,   0
STS VFS Cache:          148,      0,       0,       0,       0,   0,   0
L VFS Cache:            328,      0,       0,       0,       0,   0,   0
LTS VFS Cache:          368,      0,       0,       0,       0,   0,   0
NCLNODE:                528,      0,       0,       0,       0,   0,   0
DIRHASH:               1024,      0,       0,       0,       0,   0,   0
reference_cache:         40,      0,       0,       0,       0,   0,   0
reference_history_cache:      8,      0,       0,       0,       0,   0,   0
range_seg_cache:         64,      0,   15246,   81288,  176756,   0,   0
zio_cache:              920,      0,    2874,    3294,  146114,   0,   0
zio_link_cache:          48,      0,    2867,    4686,  145725,   0,   0
zio_buf_512:            512,      0,    4917,     221,    5752,   0,   0
zio_data_buf_512:       512,      0,     362,     247,     801,   0,   0
zio_buf_1024:          1024,      0,      95,     113,     269,   0,   0
zio_data_buf_1024:     1024,      0,     421,      99,     521,   0,   0
zio_buf_1536:          1536,      0,      35,     127,     335,   0,   0
zio_data_buf_1536:     1536,      0,     717,      85,     845,   0,   0
zio_buf_2048:          2048,      0,      45,      91,     262,   0,   0
zio_data_buf_2048:     2048,      0,     295,      51,     408,   0,   0
zio_buf_2560:          2560,      0,      11,      70,     131,   0,   0
zio_data_buf_2560:     2560,      0,     202,      32,     291,   0,   0
zio_buf_3072:          3072,      0,       4,      53,      79,   0,   0
zio_data_buf_3072:     3072,      0,     151,      29,     225,   0,   0
zio_buf_3584:          3584,      0,       5,      28,      49,   0,   0
zio_data_buf_3584:     3584,      0,     106,      11,     153,   0,   0
zio_buf_4096:          4096,      0,    3767,    1199,   45373,   0,   0
zio_data_buf_4096:     4096,      0,      97,     116,    2911,   0,   0
zio_buf_5120:          5120,      0,       2,      55,      89,   0,   0
zio_data_buf_5120:     5120,      0,     185,      21,     272,   0,   0
zio_buf_6144:          6144,      0,       5,      35,      62,   0,   0
zio_data_buf_6144:     6144,      0,     150,      13,     206,   0,   0
zio_buf_7168:          7168,      0,       2,      37,      51,   0,   0
zio_data_buf_7168:     7168,      0,     110,      10,     164,   0,   0
zio_buf_8192:          8192,      0,       2,     286,    1766,   0,   0
zio_data_buf_8192:     8192,      0,     102,       9,     192,   0,   0
zio_buf_10240:        10240,      0,       1,      49,      59,   0,   0
zio_data_buf_10240:   10240,      0,     145,      10,     215,   0,   0
zio_buf_12288:        12288,      0,       1,     176,    1011,   0,   0
zio_data_buf_12288:   12288,      0,     102,      10,     149,   0,   0
zio_buf_14336:        14336,      0,       0,      24,      27,   0,   0
zio_data_buf_14336:   14336,      0,      67,      12,     100,   0,   0
zio_buf_16384:        16384,      0,     718,      40,    2238,   0,   0
zio_data_buf_16384:   16384,      0,      57,       8,      86,   0,   0
zio_buf_20480:        20480,      0,       0,      57,     571,   0,   0
zio_data_buf_20480:   20480,      0,      59,       7,     102,   0,   0
zio_buf_24576:        24576,      0,       0,      40,     565,   0,   0
zio_data_buf_24576:   24576,      0,      44,       6,      67,   0,   0
zio_buf_28672:        28672,      0,       0,     131,    1560,   0,   0
zio_data_buf_28672:   28672,      0,      40,      13,      60,   0,   0
zio_buf_32768:        32768,      0,       0,      27,     290,   0,   0
zio_data_buf_32768:   32768,      0,      31,       5,      43,   0,   0
zio_buf_36864:        36864,      0,       1,      16,     244,   0,   0
zio_data_buf_36864:   36864,      0,      32,       3,      40,   0,   0
zio_buf_40960:        40960,      0,       0,      14,     139,   0,   0
zio_data_buf_40960:   40960,      0,      18,       1,      19,   0,   0
zio_buf_45056:        45056,      0,       0,      13,      99,   0,   0
zio_data_buf_45056:   45056,      0,      21,       3,      25,   0,   0
zio_buf_49152:        49152,      0,       0,      18,      89,   0,   0
zio_data_buf_49152:   49152,      0,      16,       3,      22,   0,   0
zio_buf_53248:        53248,      0,       1,      17,     148,   0,   0
zio_data_buf_53248:   53248,      0,      22,       3,      26,   0,   0
zio_buf_57344:        57344,      0,       0,      11,      73,   0,   0
zio_data_buf_57344:   57344,      0,      14,       1,      15,   0,   0
zio_buf_61440:        61440,      0,       0,      12,      56,   0,   0
zio_data_buf_61440:   61440,      0,      10,       3,      13,   0,   0
zio_buf_65536:        65536,      0,       0,      11,      46,   0,   0
zio_data_buf_65536:   65536,      0,       4,       1,       5,   0,   0
zio_buf_69632:        69632,      0,       0,       8,      27,   0,   0
zio_data_buf_69632:   69632,      0,       8,       1,       9,   0,   0
zio_buf_73728:        73728,      0,       0,       9,      27,   0,   0
zio_data_buf_73728:   73728,      0,       5,       0,       5,   0,   0
zio_buf_77824:        77824,      0,       0,       9,      41,   0,   0
zio_data_buf_77824:   77824,      0,       6,       1,       7,   0,   0
zio_buf_81920:        81920,      0,       0,      13,      84,   0,   0
zio_data_buf_81920:   81920,      0,       9,       1,      10,   0,   0
zio_buf_86016:        86016,      0,       0,       8,      25,   0,   0
zio_data_buf_86016:   86016,      0,       5,       3,       8,   0,   0
zio_buf_90112:        90112,      0,       0,      10,      33,   0,   0
zio_data_buf_90112:   90112,      0,       3,       1,       4,   0,   0
zio_buf_94208:        94208,      0,       0,       6,      15,   0,   0
zio_data_buf_94208:   94208,      0,       4,       1,       5,   0,   0
zio_buf_98304:        98304,      0,       1,       5,      20,   0,   0
zio_data_buf_98304:   98304,      0,       9,       0,       9,   0,   0
zio_buf_102400:      102400,      0,       0,       7,      13,   0,   0
zio_data_buf_102400: 102400,      0,       2,       1,       3,   0,   0
zio_buf_106496:      106496,      0,       0,      16,      99,   0,   0
zio_data_buf_106496: 106496,      0,      12,       2,      14,   0,   0
zio_buf_110592:      110592,      0,       0,      12,      72,   0,   0
zio_data_buf_110592: 110592,      0,       6,       1,       7,   0,   0
zio_buf_114688:      114688,      0,       0,      22,      79,   0,   0
zio_data_buf_114688: 114688,      0,       6,       0,       6,   0,   0
zio_buf_118784:      118784,      0,       0,       8,      22,   0,   0
zio_data_buf_118784: 118784,      0,       0,       0,       0,   0,   0
zio_buf_122880:      122880,      0,       0,       9,      28,   0,   0
zio_data_buf_122880: 122880,      0,       6,       0,       6,   0,   0
zio_buf_126976:      126976,      0,       0,      10,      22,   0,   0
zio_data_buf_126976: 126976,      0,       0,       0,       0,   0,   0
zio_buf_131072:      131072,      0,     160,      79,     862,   0,   0
zio_data_buf_131072: 131072,      0,     383,       8,     556,   0,   0
lz4_ctx:              16384,      0,       0,       0,       0,   0,   0
sa_cache:                80,      0,    4199,     848,    4199,   0,   0
dnode_t:               1096,      0,    4649,      31,    5833,   0,   0
dmu_buf_impl_t:         336,      0,    9678,      68,   14441,   0,   0
arc_buf_hdr_t:          328,      0,   10162,     134,   11502,   0,   0
arc_buf_t:               72,      0,    9396,     614,   12289,   0,   0
zil_lwb_cache:          192,      0,       0,       0,       0,   0,   0
zfs_znode_cache:        368,      0,    4199,     151,    4199,   0,   0
pipe:                   744,      0,       0,      55,      13,   0,   0
Mountpoints:            816,      0,      17,      23,      17,   0,   0
procdesc:               128,      0,       0,       0,       0,   0,   0
ksiginfo:               112,      0,      24,    1026,      24,   0,   0
itimer:                 352,      0,       0,       0,       0,   0,   0
KNOTE:                  128,      0,       0,       0,       0,   0,   0
socket:                 696, 2062880,       0,       0,       0,   0,   0
ipq:                     56, 125599,       0,       0,       0,   0,   0
udp_inpcb:              392, 2062880,       0,       0,       0,   0,   0
udpcb:                   16, 2062965,       0,       0,       0,   0,   0
tcp_inpcb:              392, 2062880,       0,       0,       0,   0,   0
tcpcb:                 1024, 2062880,       0,       0,       0,   0,   0
tcptw:                   88,  27810,       0,       0,       0,   0,   0
syncache:               160,  15360,       0,       0,       0,   0,   0
hostcache:              136,  15370,       0,       0,       0,   0,   0
tcpreass:                40, 251262,       0,       0,       0,   0,   0
sackhole:                32,      0,       0,       0,       0,   0,   0
sctp_ep:               1408, 2062880,       0,       0,       0,   0,   0
sctp_asoc:             2352,  40000,       0,       0,       0,   0,   0
sctp_laddr:              48,  80012,       0,       0,       0,   0,   0
sctp_raddr:             728,  80000,       0,       0,       0,   0,   0
sctp_chunk:             136, 400026,       0,       0,       0,   0,   0
sctp_readq:             104, 400026,       0,       0,       0,   0,   0
sctp_stream_msg_out:    104, 400026,       0,       0,       0,   0,   0
sctp_asconf:             40, 400059,       0,       0,       0,   0,   0
sctp_asconf_ack:         48, 400060,       0,       0,       0,   0,   0
ripcb:                  392, 2062880,       0,       0,       0,   0,   0
unpcb:                  240, 2062880,       0,       0,       0,   0,   0
rtentry:                200,      0,       0,       0,       0,   0,   0
selfd:                   56,      0,       0,       0,       0,   0,   0
SWAPMETA:               288, 8037731,       0,       0,       0,   0,   0


------------------------------------------------------------------------
vmstat -i

interrupt                          total       rate
irq6: fdc0                            22          0
irq14: ata0                           61          0
irq17: uhci0 ehci0                    54          0
irq18: uhci2+                        306          2
irq19: uhci1 ahci0+                21712        146
cpu0:timer                          3524         23
irq259: hdac0                          9          0
cpu7:timer                          2042         13
cpu1:timer                          1450          9
cpu3:timer                          3609         24
cpu2:timer                          1414          9
cpu6:timer                          1949         13
cpu4:timer                          1408          9
cpu5:timer                          1575         10
Total                              39135        264

------------------------------------------------------------------------
pstat -T

  8/2062880 files
0M/147455M swap space

------------------------------------------------------------------------
pstat -s

Device          512-blocks     Used    Avail Capacity
/dev/#C:0xc0      50331392        0 50331392     0%
/dev/ad14p2       50331392        0 50331392     0%
/dev/usb/3.2.1    50331392        0 50331392     0%
/dev/#C:0xd0      50331392        0 50331392     0%
/dev/#C:0xd8      50331392        0 50331392     0%
/dev/#C:0xe0      50331392        0 50331392     0%
Total            301988352        0 301988352     0%

------------------------------------------------------------------------
iostat

       tty            ada0             ada1             ada2             cpu
 tin  tout  KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s  us ni sy in id
   0     9 15.20 161  2.39  14.99 169  2.47  15.24 167  2.49   0  0  4  1 96

------------------------------------------------------------------------
ipcs -a

Message Queues:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP                 CBYTES                 QNUM               QBYTES        LSPID        LRPID STIME    RTIME    CTIME   

Shared Memory:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP         NATTCH        SEGSZ         CPID         LPID ATIME    DTIME    CTIME   

Semaphores:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP          NSEMS OTIME    CTIME   


------------------------------------------------------------------------
ipcs -T

msginfo:
	msgmax:        16384	(max characters in a message)
	msgmni:           40	(# of message queues)
	msgmnb:         2048	(max characters in a message queue)
	msgtql:           40	(max # of messages in system)
	msgssz:            8	(size of a message segment)
	msgseg:         2048	(# of message segments in system)

shminfo:
	shmmax:    536870912	(max shared memory segment size)
	shmmin:            1	(min shared memory segment size)
	shmmni:          192	(max number of shared memory identifiers)
	shmseg:          128	(max shared memory segments per process)
	shmall:       131072	(max amount of shared memory in pages)

seminfo:
	semmni:           50	(# of semaphore identifiers)
	semmns:          340	(# of semaphores in system)
	semmnu:          150	(# of undo structures in system)
	semmsl:          340	(max # of semaphores per id)
	semopm:          100	(max # of operations per semop call)
	semume:           50	(max # of undo entries per process)
	semusz:          632	(size in bytes of undo structure)
	semvmx:        32767	(semaphore maximum value)
	semaem:        16384	(adjust on exit max value)


------------------------------------------------------------------------
nfsstat

Client Info:
Rpc Counts:
  Getattr   Setattr    Lookup  Readlink      Read     Write    Create    Remove
        2         0         0         0         0         0         0         0
   Rename      Link   Symlink     Mkdir     Rmdir   Readdir  RdirPlus    Access
        0         0         0         0         0         0         0         0
    Mknod    Fsstat    Fsinfo  PathConf    Commit
        0         1         1         0         0
Rpc Info:
 TimedOut   Invalid X Replies   Retries  Requests
        0         0         0         0         4
Cache Info:
Attr Hits    Misses Lkup Hits    Misses BioR Hits    Misses BioW Hits    Misses
        0         0         0         0         0         0         0         0
BioRLHits    Misses BioD Hits    Misses DirE Hits    Misses Accs Hits    Misses
        0         0         0         0         0         0         0         0

Server Info:
  Getattr   Setattr    Lookup  Readlink      Read     Write    Create    Remove
        0         0         0         0         0         0         0         0
   Rename      Link   Symlink     Mkdir     Rmdir   Readdir  RdirPlus    Access
        0         0         0         0         0         0         0         0
    Mknod    Fsstat    Fsinfo  PathConf    Commit
        0         0         0         0         0
Server Ret-Failed
                0
Server Faults
            0
Server Cache Stats:
   Inprog      Idem  Non-idem    Misses
        0         0         0         0
Server Write Gathering:
 WriteOps  WriteRPC   Opsaved
        0         0         0

------------------------------------------------------------------------
netstat -s

tcp:
	0 packets sent
		0 data packets (0 bytes)
		0 data packets (0 bytes) retransmitted
		0 data packets unnecessarily retransmitted
		0 resends initiated by MTU discovery
		0 ack-only packets (0 delayed)
		0 URG only packets
		0 window probe packets
		0 window update packets
		0 control packets
	0 packets received
		0 acks (for 0 bytes)
		0 duplicate acks
		0 acks for unsent data
		0 packets (0 bytes) received in-sequence
		0 completely duplicate packets (0 bytes)
		0 old duplicate packets
		0 packets with some dup. data (0 bytes duped)
		0 out-of-order packets (0 bytes)
		0 packets (0 bytes) of data after window
		0 window probes
		0 window update packets
		0 packets received after close
		0 discarded for bad checksums
		0 discarded for bad header offset fields
		0 discarded because packet too short
		0 discarded due to memory problems
	0 connection requests
	0 connection accepts
	0 bad connection attempts
	0 listen queue overflows
	0 ignored RSTs in the windows
	0 connections established (including accepts)
	0 connections closed (including 0 drops)
		0 connections updated cached RTT on close
		0 connections updated cached RTT variance on close
		0 connections updated cached ssthresh on close
	0 embryonic connections dropped
	0 segments updated rtt (of 0 attempts)
	0 retransmit timeouts
		0 connections dropped by rexmit timeout
	0 persist timeouts
		0 connections dropped by persist timeout
	0 Connections (fin_wait_2) dropped because of timeout
	0 keepalive timeouts
		0 keepalive probes sent
		0 connections dropped by keepalive
	0 correct ACK header predictions
	0 correct data packet header predictions
	0 syncache entries added
		0 retransmitted
		0 dupsyn
		0 dropped
		0 completed
		0 bucket overflow
		0 cache overflow
		0 reset
		0 stale
		0 aborted
		0 badack
		0 unreach
		0 zone failures
	0 cookies sent
	0 cookies received
	0 hostcache entries added
		0 bucket overflow
	0 SACK recovery episodes
	0 segment rexmits in SACK recovery episodes
	0 byte rexmits in SACK recovery episodes
	0 SACK options (SACK blocks) received
	0 SACK options (SACK blocks) sent
	0 SACK scoreboard overflow
	0 packets with ECN CE bit set
	0 packets with ECN ECT(0) bit set
	0 packets with ECN ECT(1) bit set
	0 successful ECN handshakes
	0 times ECN reduced the congestion window
udp:
	0 datagrams received
	0 with incomplete header
	0 with bad data length field
	0 with bad checksum
	0 with no checksum
	0 dropped due to no socket
	0 broadcast/multicast datagrams undelivered
	0 dropped due to full socket buffers
	0 not for hashed pcb
	0 delivered
	0 datagrams output
	0 times multicast source filter matched
ip:
	0 total packets received
	0 bad header checksums
	0 with size smaller than minimum
	0 with data size < data length
	0 with ip length > max ip packet size
	0 with header length < data size
	0 with data length < header length
	0 with bad options
	0 with incorrect version number
	0 fragments received
	0 fragments dropped (dup or out of space)
	0 fragments dropped after timeout
	0 packets reassembled ok
	0 packets for this host
	0 packets for unknown/unsupported protocol
	0 packets forwarded (0 packets fast forwarded)
	0 packets not forwardable
	0 packets received for unknown multicast group
	0 redirects sent
	0 packets sent from this host
	0 packets sent with fabricated ip header
	0 output packets dropped due to no bufs, etc.
	0 output packets discarded due to no route
	0 output datagrams fragmented
	0 fragments created
	0 datagrams that can't be fragmented
	0 tunneling packets that can't find gif
	0 datagrams with bad address in header
icmp:
	0 calls to icmp_error
	0 errors not generated in response to an icmp message
	0 messages with bad code fields
	0 messages less than the minimum length
	0 messages with bad checksum
	0 messages with bad length
	0 multicast echo requests ignored
	0 multicast timestamp requests ignored
	0 message responses generated
	0 invalid return addresses
	0 no return routes
igmp:
	0 messages received
	0 messages received with too few bytes
	0 messages received with wrong TTL
	0 messages received with bad checksum
	0 V1/V2 membership queries received
	0 V3 membership queries received
	0 membership queries received with invalid field(s)
	0 general queries received
	0 group queries received
	0 group-source queries received
	0 group-source queries dropped
	0 membership reports received
	0 membership reports received with invalid field(s)
	0 membership reports received for groups to which we belong
	0 V3 reports received without Router Alert
	0 membership reports sent
arp:
	0 ARP requests sent
	0 ARP replies sent
	0 ARP requests received
	0 ARP replies received
	0 ARP packets received
	0 total packets dropped due to no ARP entry
	0 ARP entrys timed out
	0 Duplicate IPs seen
ip6:
	0 total packets received
	0 with size smaller than minimum
	0 with data size < data length
	0 with bad options
	0 with incorrect version number
	0 fragments received
	0 fragments dropped (dup or out of space)
	0 fragments dropped after timeout
	0 fragments that exceeded limit
	0 packets reassembled ok
	0 packets for this host
	0 packets forwarded
	0 packets not forwardable
	0 redirects sent
	0 packets sent from this host
	0 packets sent with fabricated ip header
	0 output packets dropped due to no bufs, etc.
	0 output packets discarded due to no route
	0 output datagrams fragmented
	0 fragments created
	0 datagrams that can't be fragmented
	0 packets that violated scope rules
	0 multicast packets which we don't join
	Mbuf statistics:
		0 one mbuf
		0 one ext mbuf
		0 two or more ext mbuf
	0 packets whose headers are not contiguous
	0 tunneling packets that can't find gif
	0 packets discarded because of too many headers
	0 failures of source address selection
	Source addresses selection rule applied:
icmp6:
	0 calls to icmp6_error
	0 errors not generated in response to an icmp6 message
	0 errors not generated because of rate limitation
	0 messages with bad code fields
	0 messages < minimum length
	0 bad checksums
	0 messages with bad length
	Histogram of error messages to be generated:
		0 no route
		0 administratively prohibited
		0 beyond scope
		0 address unreachable
		0 port unreachable
		0 packet too big
		0 time exceed transit
		0 time exceed reassembly
		0 erroneous header field
		0 unrecognized next header
		0 unrecognized option
		0 redirect
		0 unknown
	0 message responses generated
	0 messages with too many ND options
	0 messages with bad ND options
	0 bad neighbor solicitation messages
	0 bad neighbor advertisement messages
	0 bad router solicitation messages
	0 bad router advertisement messages
	0 bad redirect messages
	0 path MTU changes
rip6:
	0 messages received
	0 checksum calculations on inbound
	0 messages with bad checksum
	0 messages dropped due to no socket
	0 multicast messages dropped due to no socket
	0 messages dropped due to full socket buffers
	0 delivered
	0 datagrams output

------------------------------------------------------------------------
netstat -m

netstat: invalid address (0x0)
1/144/145 mbufs in use (current/cache/total)
18446744073709551606/10/0/4018860 mbuf clusters in use (current/cache/total/max)
0/10 mbuf+clusters out of packet secondary zone in use (current/cache)
0/0/0/2009429 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/1786158 9k jumbo clusters in use (current/cache/total/max)
0/0/0/1339616 16k jumbo clusters in use (current/cache/total/max)
18014398509481964K/56K/36K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)

------------------------------------------------------------------------
netstat -anr

Routing tables

Internet:
Destination        Gateway            Flags    Netif Expire

Internet6:
Destination                       Gateway                       Flags      Netif Expire

------------------------------------------------------------------------
netstat -anA


------------------------------------------------------------------------
netstat -aL


------------------------------------------------------------------------
fstat

fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 7 at 0x200007fffffffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 7 at 0x200007fffffffff
fstat: can't read file 8 at 0x4000000001fffff
fstat: can't read file 10 at 0x780000ffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read file 1 at 0x200007fffffffff
fstat: can't read file 2 at 0x4000000001fffff
fstat: can't read file 4 at 0x780000ffff
fstat: can't read file 7 at 0x200007fffffffff
fstat: can't read file 8 at 0x4000000001fffff
fstat: can't read file 10 at 0x780000ffff
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
fstat: can't read znode_phys at 0x1
USER     CMD          PID   FD MOUNT      INUM MODE         SZ|DV R/W
root     zfs           75 root -         -       error    -
root     zfs           75   wd -         -       error    -
root     zfs           75 text -         -       error    -
root     zfs           75 ctty /dev         13 crw-------  console rw
root     zfs           75    0 /dev         13 crw-------  console rw
root     zfs           75    6 /dev         13 crw-------  console rw
root     sh            72 root -         -       error    -
root     sh            72   wd -         -       error    -
root     sh            72 text -         -       error    -
root     sh            72 ctty /dev         13 crw-------  console rw
root     sh            72    0 /dev         13 crw-------  console rw
root     sh            72    6 /dev         13 crw-------  console rw
root     sh            24 root -         -       error    -
root     sh            24   wd -         -       error    -
root     sh            24 text -         -       error    -
root     sh            24 ctty /dev         13 crw-------  console rw
root     sh            24    0 /dev         13 crw-------  console rw
root     sh            24    6 /dev         13 crw-------  console rw
root     init           1 root -         -       error    -
root     init           1   wd -         -       error    -
root     init           1 text -         -       error    -
root     kernel         0 root -         -       error    -
root     kernel         0   wd -         -       error    -

------------------------------------------------------------------------
dmesg

Copyright (c) 1992-2014 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.0-CURRENT #3 r261166: Sat Jan 25 14:55:24 CST 2014
    root_at_borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64
FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610
info: [drm] Initialized drm 1.1.0 20060810
CPU: Intel(R) Xeon(R) CPU           E5410  _at_ 2.33GHz (2327.55-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x10676  Family=0x6  Model=0x17  Stepping=6
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0xce3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1>
  AMD Features=0x20100800<SYSCALL,NX,LM>
  AMD Features2=0x1<LAHF>
  TSC: P-state invariant, performance statistics
real memory  = 68719476736 (65536 MB)
avail memory = 65641644032 (62600 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <PTLTD  	 APIC  >
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 2 package(s) x 4 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
 cpu2 (AP): APIC ID:  2
 cpu3 (AP): APIC ID:  3
 cpu4 (AP): APIC ID:  4
 cpu5 (AP): APIC ID:  5
 cpu6 (AP): APIC ID:  6
 cpu7 (AP): APIC ID:  7
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 24-47 on motherboard
random: <Software, Yarrow> initialized
kbd1 at kbdmux0
cryptosoft0: <software crypto> on motherboard
acpi0: <PTLTD   RSDT> on motherboard
acpi0: Power Button (fixed)
unknown: I/O range not supported
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
cpu4: <ACPI CPU> on acpi0
cpu5: <ACPI CPU> on acpi0
cpu6: <ACPI CPU> on acpi0
cpu7: <ACPI CPU> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff irq 0,8 on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 350
Event timer "HPET1" frequency 14318180 Hz quality 340
Event timer "HPET2" frequency 14318180 Hz quality 340
atrtc0: <AT realtime clock> port 0x70-0x71 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0
pci1: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> irq 16 at device 0.0 on pci1
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> irq 16 at device 0.0 on pci2
pci3: <ACPI PCI bus> on pcib3
pcib4: <ACPI PCI-PCI bridge> at device 0.0 on pci3
pci4: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> at device 0.2 on pci3
pci5: <ACPI PCI bus> on pcib5
pcib6: <ACPI PCI-PCI bridge> irq 18 at device 2.0 on pci2
pci6: <ACPI PCI bus> on pcib6
em0: <Intel(R) PRO/1000 Network Connection 7.3.8> port 0x2000-0x201f mem 0xd9220000-0xd923ffff,0xd9200000-0xd921ffff irq 18 at device 0.0 on pci6
em0: Using an MSI interrupt
em0: Ethernet address: 00:30:48:f2:29:9c
em1: <Intel(R) PRO/1000 Network Connection 7.3.8> port 0x2020-0x203f mem 0xd9260000-0xd927ffff,0xd9240000-0xd925ffff irq 19 at device 0.1 on pci6
em1: Using an MSI interrupt
em1: Ethernet address: 00:30:48:f2:29:9d
pcib7: <ACPI PCI-PCI bridge> at device 0.3 on pci1
pci7: <ACPI PCI bus> on pcib7
pcib8: <ACPI PCI-PCI bridge> at device 4.0 on pci0
pci8: <ACPI PCI bus> on pcib8
vgapci0: <VGA-compatible display> port 0x3000-0x307f mem 0xd8000000-0xd8ffffff,0xc0000000-0xc7ffffff,0xc8000000-0xc9ffffff irq 16 at device 0.0 on pci8
nvidia0: <GeForce GT 610> on vgapci0
vgapci0: child nvidia0 requested pci_enable_io
vgapci0: child nvidia0 requested pci_enable_io
hdac0: <NVIDIA GF119 HDA Controller> mem 0xd9000000-0xd9003fff irq 17 at device 0.1 on pci8
pcib9: <ACPI PCI-PCI bridge> at device 6.0 on pci0
pci9: <ACPI PCI bus> on pcib9
pci0: <base peripheral> at device 8.0 (no driver attached)
pcib10: <ACPI PCI-PCI bridge> irq 17 at device 28.0 on pci0
pci10: <ACPI PCI bus> on pcib10
pcib11: <PCI-PCI bridge> irq 16 at device 0.0 on pci10
pci11: <PCI bus> on pcib11
pcm0: <Envy24HT audio (Generic)> port 0x4080-0x409f,0x4000-0x407f irq 16 at device 0.0 on pci11
pcm0: system configuration
  SubVendorID: 0x1412, SubDeviceID: 0x2403
  XIN2 Clock Source: 24.576MHz(96kHz*256)
  MPU-401 UART(s) #: not implemented
  ADC #: 1 and SPDIF receiver connected
  DAC #: 4
  Multi-track converter type: AC'97(SDATA_OUT:packed)
  S/PDIF(IN/OUT): 1/1 ID# 0x00
  GPIO(mask/dir/state): 0xff/0xff/0xff
uhci0: <Intel 631XESB/632XESB/3100 USB controller USB-1> port 0x1800-0x181f irq 17 at device 29.0 on pci0
usbus0 on uhci0
uhci1: <Intel 631XESB/632XESB/3100 USB controller USB-2> port 0x1820-0x183f irq 19 at device 29.1 on pci0
usbus1 on uhci1
uhci2: <Intel 631XESB/632XESB/3100 USB controller USB-3> port 0x1840-0x185f irq 18 at device 29.2 on pci0
usbus2 on uhci2
ehci0: <Intel 63XXESB USB 2.0 controller> mem 0xd9600400-0xd96007ff irq 17 at device 29.7 on pci0
usbus3: EHCI version 1.0
usbus3 on ehci0
pcib12: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci12: <ACPI PCI bus> on pcib12
vgapci1: <VGA-compatible display> port 0x5000-0x50ff mem 0xd0000000-0xd7ffffff,0xd9300000-0xd930ffff irq 18 at device 1.0 on pci12
drmn1: <ATI ES1000 RN50> on vgapci1
info: [drm] RADEON_IS_PCI
info: [drm] initializing kernel modesetting (RV100 0x1002:0x515E 0x15D9:0x8080).
info: [drm] register mmio base: 0xD9300000
info: [drm] register mmio size: 65536
info: [drm] radeon_atrm_get_bios: ===> Try ATRM...
info: [drm] radeon_atrm_get_bios: pci_find_class() found: 0:8:0:0, vendor=10de, device=104a
info: [drm] radeon_atrm_get_bios: Get ACPI device handle
info: [drm] radeon_acpi_vfct_bios: ===> Try VFCT...
info: [drm] radeon_acpi_vfct_bios: Get "VFCT" ACPI table
info: [drm] radeon_acpi_vfct_bios: Failed to get "VFCT" table: AE_NOT_FOUND
info: [drm] igp_read_bios_from_vram: ===> Try IGP's VRAM...
info: [drm] igp_read_bios_from_vram: VRAM base address: 0xd0000000
info: [drm] igp_read_bios_from_vram: Map address: 0xfffff800d0000000 (262144 bytes)
info: [drm] igp_read_bios_from_vram: Incorrect BIOS signature: 0x0000
info: [drm] radeon_read_bios: ===> Try PCI Expansion ROM...
info: [drm] radeon_read_bios: Map address: 0xfffff800000c0000 (131072 bytes)
drmn1: info: VRAM: 128M 0x00000000D0000000 - 0x00000000D7FFFFFF (16M used)
drmn1: info: GTT: 512M 0x00000000B0000000 - 0x00000000CFFFFFFF
info: [drm] Supports vblank timestamp caching Rev 1 (10.10.2010).
info: [drm] Driver supports precise vblank timestamp query.
info: [drm] radeon: irq initialized.
info: [drm] Detected VRAM RAM=128M, BAR=128M
info: [drm] RAM width 64bits SDR
[TTM] Zone  kernel: Available graphics memory: 33006090 kiB
[TTM] Zone   dma32: Available graphics memory: 2097152 kiB
[TTM] Initializing pool allocator
info: [drm] radeon: 16M of VRAM memory ready
info: [drm] radeon: 512M of GTT memory ready.
info: [drm] GART: num cpu pages 131072, num gpu pages 131072
info: [drm] PCI GART of 512M enabled (table at 0x0000000011200000).
drmn1: info: WB disabled
drmn1: info: fence driver on ring 0 use gpu addr 0x00000000b0000000 and cpu addr 0x0xfffff8001083c000
info: [drm] Loading R100 Microcode
info: [drm] radeon: ring at 0x00000000B0001000
info: [drm] ring test succeeded in 1 usecs
info: [drm] ib test succeeded in 0 usecs
info: [drm] radeon_device_init: Taking over the fictitious range 0xd0000000-0xd4000000
iicbus0: <Philips I2C bus> on iicbb0 addr 0xff
iic0: <I2C generic I/O> on iicbus0
iicbus1: <Philips I2C bus> on iicbb1 addr 0xff
iic1: <I2C generic I/O> on iicbus1
iicbus2: <Philips I2C bus> on iicbb2 addr 0xff
iic2: <I2C generic I/O> on iicbus2
iicbus3: <Philips I2C bus> on iicbb3 addr 0xff
iic3: <I2C generic I/O> on iicbus3
info: [drm] Radeon Display Connectors
info: [drm] Connector 0:
info: [drm]   VGA-1
info: [drm]   DDC: 0x60 0x60 0x60 0x60 0x60 0x60 0x60 0x60
info: [drm]   Encoders:
info: [drm]     CRT1: INTERNAL_DAC1
info: [drm] Connector 1:
info: [drm]   DVI-I-1
info: [drm]   HPD2
info: [drm]   DDC: 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c
info: [drm]   Encoders:
info: [drm]     CRT2: INTERNAL_DAC2
info: [drm]     DFP2: INTERNAL_DVO1
composite sync not supported
composite sync not supported
info: [drm] fb mappable at 0xD0040000
info: [drm] vram apper at 0xD0000000
info: [drm] size 2076672
info: [drm] fb depth is 8
info: [drm]    pitch is 1920
fbd1 on drmn1
vt_allocate: Replace existing VT driver.
info: [drm] Initialized radeon 2.29.0 20080528
vgapci1: Boot video device
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel 63XXESB2 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1860-0x186f at device 31.1 on pci0
ata0: <ATA channel> at channel 0 on atapci0
ahci0: <Intel ESB2 AHCI SATA controller> port 0x18a0-0x18a7,0x1874-0x1877,0x1878-0x187f,0x1870-0x1873,0x1880-0x189f mem 0xd9600800-0xd9600bff irq 19 at device 31.2 on pci0
ahci0: AHCI v1.10 with 6 3Gbps ports, Port Multiplier supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich5: <AHCI channel> at channel 5 on ahci0
ichsmb0: <Intel 631xESB/6321ESB (ESB2) SMBus controller> port 0x1100-0x111f irq 19 at device 31.3 on pci0
smbus0: <System Management Bus> on ichsmb0
acpi_button0: <Power Button> on acpi0
ipmi0: <IPMI System Interface> port 0xca2-0xca3 on acpi0
ipmi0: KCS mode found at io 0xca2 on acpi
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
ppc0: <Parallel port> port 0x378-0x37f,0x778-0x77f irq 7 drq 3 on acpi0
ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode
ppc0: FIFO with 16/16/9 bytes threshold
ppbus0: <Parallel port bus> on ppc0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
ichwd0 on isa0
orm0: <ISA Option ROM> at iomem 0xc0000-0xcafff on isa0
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
coretemp1: <CPU On-Die Thermal Sensors> on cpu1
est1: <Enhanced SpeedStep Frequency Control> on cpu1
p4tcc1: <CPU Frequency Thermal Control> on cpu1
coretemp2: <CPU On-Die Thermal Sensors> on cpu2
est2: <Enhanced SpeedStep Frequency Control> on cpu2
p4tcc2: <CPU Frequency Thermal Control> on cpu2
coretemp3: <CPU On-Die Thermal Sensors> on cpu3
est3: <Enhanced SpeedStep Frequency Control> on cpu3
p4tcc3: <CPU Frequency Thermal Control> on cpu3
coretemp4: <CPU On-Die Thermal Sensors> on cpu4
est4: <Enhanced SpeedStep Frequency Control> on cpu4
p4tcc4: <CPU Frequency Thermal Control> on cpu4
coretemp5: <CPU On-Die Thermal Sensors> on cpu5
est5: <Enhanced SpeedStep Frequency Control> on cpu5
p4tcc5: <CPU Frequency Thermal Control> on cpu5
coretemp6: <CPU On-Die Thermal Sensors> on cpu6
est6: <Enhanced SpeedStep Frequency Control> on cpu6
p4tcc6: <CPU Frequency Thermal Control> on cpu6
coretemp7: <CPU On-Die Thermal Sensors> on cpu7
est7: <Enhanced SpeedStep Frequency Control> on cpu7
p4tcc7: <CPU Frequency Thermal Control> on cpu7
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 1.000 msec
vboxdrv: fAsync=0 offMin=0x387 offMax=0x50f
hdacc0: <NVIDIA (0x001c) HDA CODEC> at cad 0 on hdac0
hdaa0: <NVIDIA (0x001c) Audio Function Group> at nid 1 on hdacc0
pcm1: <NVIDIA (0x001c) (HDMI/DP 8ch)> at nid 4 on hdaa0
pcm2: <NVIDIA (0x001c) (HDMI/DP 8ch)> at nid 5 on hdaa0
random: unblocking device.
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 480Mbps High Speed USB v2.0
ugen3.1: <Intel> at usbus3
uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3
ugen2.1: <Intel> at usbus2
uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
ugen1.1: <Intel> at usbus1
uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
ugen0.1: <Intel> at usbus0
uhub3: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ipmi0: IPMI device rev. 1, firmware rev. 1.64, version 2.0
ata0: DMA limited to UDMA33, controller found non-ATA66 cable
ipmi0: Number of channels 8
ipmi0: Attached watchdog
uhub2: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub3: 2 ports with 2 removable, self powered
ada0 at ahcich0 bus 0 scbus1 target 0 lun 0
ada0: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
ada0: Serial Number 5YDA1ZL4
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada0: quirks=0x1<4K>
ada0: Previously was known as ad4
ada1 at ahcich1 bus 0 scbus2 target 0 lun 0
ada1: <ST2000DL003-9VT166 CC3D> ATA-8 SATA 3.x device
ada1: Serial Number 5YD6FPLG
ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada1: quirks=0x1<4K>
ada1: Previously was known as ad6
ada2 at ahcich2 bus 0 scbus3 target 0 lun 0
ada2: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
ada2: Serial Number 5YDA3PC5
ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada2: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada2: quirks=0x1<4K>
ada2: Previously was known as ad8
ada3 at ahcich3 bus 0 scbus4 target 0 lun 0
ada3: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
ada3: Serial Number 5YD9Y0P4
ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada3: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada3: quirks=0x1<4K>
ada3: Previously was known as ad10
ada4 at ahcich4 bus 0 scbus5 target 0 lun 0
ada4: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
ada4: Serial Number 5YDA1R5W
ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada4: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada4: quirks=0x1<4K>
ada4: Previously was known as ad12
ada5 at ahcich5 bus 0 scbus6 target 0 lun 0
ada5: <ST2000DL003-9VT166 CC3D> ATA-8 SATA 3.x device
ada5: Serial Number 5YD5RBS8
ada5: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada5: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada5: quirks=0x1<4K>
ada5: Previously was known as ad14
cd0 at ata0 bus 0 scbus0 target 0 lun 0
cd0: <Memorex DVD+-RAM 510L v1 MWS7> Removable CD-ROM SCSI-0 device 
cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present
Netvsc initializing... done!
SMP: AP CPU #7 Launched!
SMP: AP CPU #1 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #6 Launched!
SMP: AP CPU #4 Launched!
SMP: AP CPU #5 Launched!
Root mount waiting for: usbus3
uhub0: 6 ports with 6 removable, self powered
Root mount waiting for: usbus3
Root mount waiting for: usbus3
ugen3.2: <Peppercon AG> at usbus3
ukbd0: <Peppercon AG Multidevice, class 0/0, rev 2.00/0.01, addr 2> on usbus3
kbd2 at ukbd0
Trying to mount root from zfs:zroot/ROOT/default []...
ugen0.2: <American Power Conversion> at usbus0
ugen1.2: <Prolific Technology Inc.> at usbus1
Setting hostuuid: 53d19f64-d663-a017-8922-0030488e9ff3.
Setting hostid: 0xf53a926e.
Entropy harvesting: interrupts ethernet point_to_point swi.
Starting file system checks:
Mounting local file systems:.
panic: solaris assert: l->l_phys->l_hdr.lh_pad1 == 0 (0xffff000000000000 == 0x0), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c, line: 479
cpuid = 7
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe100c55d350
kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe100c55d400
vpanic() at vpanic+0x126/frame 0xfffffe100c55d440
panic() at panic+0x43/frame 0xfffffe100c55d4a0
assfail3() at assfail3+0x2f/frame 0xfffffe100c55d4c0
zap_get_leaf_byblk() at zap_get_leaf_byblk+0x2ea/frame 0xfffffe100c55d510
zap_deref_leaf() at zap_deref_leaf+0xd1/frame 0xfffffe100c55d560
fzap_length() at fzap_length+0x31/frame 0xfffffe100c55d5d0
zap_length_uint64() at zap_length_uint64+0xbf/frame 0xfffffe100c55d630
ddt_zap_lookup() at ddt_zap_lookup+0x3d/frame 0xfffffe100c55d790
ddt_lookup() at ddt_lookup+0x17e/frame 0xfffffe100c55d960
zio_ddt_write() at zio_ddt_write+0xf2/frame 0xfffffe100c55dad0
zio_execute() at zio_execute+0x1eb/frame 0xfffffe100c55db30
taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe100c55db80
taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe100c55dbb0
fork_exit() at fork_exit+0x84/frame 0xfffffe100c55dbf0
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe100c55dbf0
--- trap 0, rip = 0, rsp = 0xfffffe100c55dcb0, rbp = 0 ---
KDB: enter: panic
Uptime: 23s
Dumping 2330 out of 64465 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%

------------------------------------------------------------------------
kernel config

options	CONFIG_AUTOGENERATED
ident	VT-LER
machine	amd64
cpu	HAMMER
makeoptions	WITH_CTF=1
makeoptions	DEBUG=-g
options	ZFS
options	XENHVM
options	USB_DEBUG
options	ATH_ENABLE_11N
options	AH_AR5416_INTERRUPT_MITIGATION
options	AH_SUPPORT_AR5416
options	IEEE80211_SUPPORT_MESH
options	IEEE80211_AMPDU_AGE
options	IEEE80211_DEBUG
options	SC_PIXEL_MODE
options	VESA
options	AHD_REG_PRETTY_PRINT
options	AHC_REG_PRETTY_PRINT
options	ATA_STATIC_ID
options	ACPI_DMAR
options	SMP
options	MALLOC_DEBUG_MAXZONES=8
options	INVARIANT_SUPPORT
options	INVARIANTS
options	DEADLKRES
options	GDB
options	DDB
options	KDB_TRACE
options	KDB
options	INCLUDE_CONFIG_FILE
options	DDB_CTF
options	KDTRACE_HOOKS
options	KDTRACE_FRAME
options	MAC
options	CAPABILITIES
options	CAPABILITY_MODE
options	AUDIT
options	HWPMC_HOOKS
options	KBD_INSTALL_CDEV
options	PRINTF_BUFR_SIZE=128
options	_KPOSIX_PRIORITY_SCHEDULING
options	SYSVSEM
options	SYSVMSG
options	SYSVSHM
options	STACK
options	KTRACE
options	SCSI_DELAY=5000
options	COMPAT_FREEBSD7
options	COMPAT_FREEBSD6
options	COMPAT_FREEBSD5
options	COMPAT_FREEBSD4
options	COMPAT_FREEBSD32
options	GEOM_LABEL
options	GEOM_RAID
options	GEOM_PART_GPT
options	PSEUDOFS
options	PROCFS
options	CD9660
options	MSDOSFS
options	NFS_ROOT
options	NFSLOCKD
options	NFSD
options	NFSCL
options	MD_ROOT
options	QUOTA
options	UFS_GJOURNAL
options	UFS_DIRHASH
options	UFS_ACL
options	SOFTUPDATES
options	FFS
options	SCTP
options	TCP_OFFLOAD
options	INET6
options	INET
options	PREEMPTION
options	SCHED_ULE
options	NEW_PCIB
options	GEOM_PART_MBR
options	GEOM_PART_EBR_COMPAT
options	GEOM_PART_EBR
options	GEOM_PART_BSD
device	isa
device	mem
device	io
device	uart_ns8250
device	cpufreq
device	acpi
device	pci
device	fdc
device	ahci
device	ata
device	mvs
device	siis
device	ahc
device	ahd
device	esp
device	hptiop
device	isp
device	mpt
device	mps
device	sym
device	trm
device	adv
device	adw
device	aic
device	bt
device	isci
device	scbus
device	ch
device	da
device	sa
device	cd
device	pass
device	ses
device	amr
device	arcmsr
device	ciss
device	dpt
device	hptmv
device	hptnr
device	hptrr
device	hpt27xx
device	iir
device	ips
device	mly
device	twa
device	tws
device	aac
device	aacp
device	aacraid
device	ida
device	mfi
device	mlx
device	twe
device	atkbdc
device	atkbd
device	psm
device	kbdmux
device	splash
device	agp
device	cbb
device	pccard
device	cardbus
device	uart
device	ppc
device	ppbus
device	lpt
device	ppi
device	puc
device	bxe
device	de
device	em
device	igb
device	ixgbe
device	le
device	ti
device	txp
device	vx
device	miibus
device	ae
device	age
device	alc
device	ale
device	bce
device	bfe
device	bge
device	cas
device	dc
device	et
device	fxp
device	gem
device	hme
device	jme
device	lge
device	msk
device	nfe
device	nge
device	pcn
device	re
device	rl
device	sf
device	sge
device	sis
device	sk
device	ste
device	stge
device	tl
device	tx
device	vge
device	vr
device	wb
device	xl
device	cs
device	ed
device	ex
device	ep
device	fe
device	sn
device	xe
device	wlan
device	wlan_wep
device	wlan_ccmp
device	wlan_tkip
device	wlan_amrr
device	an
device	ath
device	ath_pci
device	ath_hal
device	ath_rate_sample
device	ipw
device	iwi
device	iwn
device	malo
device	mwl
device	ral
device	wi
device	wpi
device	loop
device	random
device	padlock_rng
device	rdrand_rng
device	ether
device	vlan
device	tun
device	md
device	gif
device	faith
device	firmware
device	bpf
device	uhci
device	ohci
device	ehci
device	xhci
device	usb
device	ukbd
device	umass
device	sound
device	snd_cmi
device	snd_csa
device	snd_emu10kx
device	snd_es137x
device	snd_hda
device	snd_ich
device	snd_via8233
device	mmc
device	mmcsd
device	sdhci
device	virtio
device	virtio_pci
device	vtnet
device	virtio_blk
device	virtio_scsi
device	virtio_balloon
device	hyperv
device	xenpci
device	vmx
device	vt
device	vt_vga

------------------------------------------------------------------------
ddb capture buffer


-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 214-642-9640                 E-Mail: ler_at_lerctr.org
US Mail: 108 Turvey Cove, Hutto, TX 78634-5688
Received on Sat Jan 25 2014 - 20:30:16 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:46 UTC