Re: LOR in r295717M

On Mon, 14 Mar 2016 15:16:06 -0700 "Robison, Dave"
<david.robison_at_fisglobal.com> wrote

> Hope this is the right place to take this. I can provide more info as
> requested. 
Hello, Dave.
I reported nearly identical LOR's last week, and was
told they're, ahem... "normal" see; harmless.
My suggestion; rebuild your kernel, and remove WITNESS from
your KERNCONF.

--Chris

>
> 
> 
> lock order reversal:
>  1st 0xfffffe03e1823700 bufwait (bufwait) _at_ /usr/src/sys/kern/vfs_bio.c:3488
>  2nd 0xfffff80005286200 dirhash (dirhash) _at_
> /usr/src/sys/ufs/ufs/ufs_dirhash.c:281 stack backtrace:
> #0 0xffffffff80a7fc60 at witness_debugger+0x70
> #1 0xffffffff80a7fb61 at witness_checkorder+0xe71
> #2 0xffffffff80a2d152 at _sx_xlock+0x72
> #3 0xffffffff80cc562d at ufsdirhash_add+0x3d
> #4 0xffffffff80cc84df at ufs_direnter+0x62f
> #5 0xffffffff80cd1843 at ufs_makeinode+0x5f3
> #6 0xffffffff80ccd3ed at ufs_create+0x2d
> #7 0xffffffff80fbb561 at VOP_CREATE_APV+0xf1
> #8 0xffffffff80ae6a18 at vn_open_cred+0x2f8
> #9 0xffffffff80adfd9c at kern_openat+0x25c
> #10 0xffffffff80e6f15b at amd64_syscall+0x2db
> #11 0xffffffff80e4ed5b at Xfast_syscall+0xfb
> 
> 
> lock order reversal:
>  1st 0xfffff80021da2d50 ufs (ufs) _at_ /usr/src/sys/kern/vfs_syscalls.c:3400
>  2nd 0xfffffe03e1823700 bufwait (bufwait) _at_
> /usr/src/sys/ufs/ffs/ffs_vnops.c:263  3rd 0xfffff80025d1ed50 ufs (ufs) _at_
> /usr/src/sys/kern/vfs_subr.c:2476 stack backtrace:
> #0 0xffffffff80a7fc60 at witness_debugger+0x70
> #1 0xffffffff80a7fb61 at witness_checkorder+0xe71
> #2 0xffffffff809ff22b at __lockmgr_args+0xd3b
> #3 0xffffffff80cc06b6 at ffs_lock+0xa6
> #4 0xffffffff80fbdb00 at VOP_LOCK1_APV+0x100
> #5 0xffffffff80ae71ba at _vn_lock+0x9a
> #6 0xffffffff80ad7783 at vget+0x63
> #7 0xffffffff80aca08c at vfs_hash_get+0xcc
> #8 0xffffffff80cbbda0 at ffs_vgetf+0x40
> #9 0xffffffff80cb3731 at softdep_sync_buf+0xad1
> #10 0xffffffff80cc1296 at ffs_syncvnode+0x256
> #11 0xffffffff80cc0550 at ffs_fsync+0x20
> #12 0xffffffff80fbcbc7 at VOP_FSYNC_APV+0xf7
> #13 0xffffffff80ae3e30 at sys_fsync+0x170
> #14 0xffffffff80e6f15b at amd64_syscall+0x2db
> #15 0xffffffff80e4ed5b at Xfast_syscall+0xfb
> 
> 
> dmesg info:
> 
> daver_at_hvm1a:~ % cat /var/run/dmesg.boot
> Copyright (c) 1992-2016 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
>         The Regents of the University of California. All rights reserved.
> FreeBSD is a registered trademark of The FreeBSD Foundation.
> FreeBSD 11.0-CURRENT #0 r295717M: Mon Feb 22 12:24:00 PST 2016
>     root_at_hvm1a.dev.vicor.com:/usr/obj/usr/src/sys/GENERIC amd64
> FreeBSD clang version 3.7.1 (tags/RELEASE_371/final 255217) 20151225
> WARNING: WITNESS option enabled, expect reduced performance.
> VT(vga): resolution 640x480
> CPU: Intel(R) Xeon(R) CPU            5150  _at_ 2.66GHz (2660.06-MHz K8-class
> CPU)   Origin="GenuineIntel"  Id=0x6f6  Family=0x6  Model=0xf  Stepping=6
>  
> Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,
> CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>  
> Features2=0x4e3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA
> >   AMD Features=0x20100800<SYSCALL,NX,LM>
>   AMD Features2=0x1<LAHF>
>   VT-x: (disabled in BIOS) HLT,PAUSE
>   TSC: P-state invariant, performance statistics
> real memory  = 17179869184 (16384 MB)
> avail memory = 16553672704 (15786 MB)
> Event timer "LAPIC" quality 400
> ACPI APIC Table: <INTEL  S5000PAL>
> FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
> FreeBSD/SMP: 1 package(s) x 2 core(s)
>  cpu0 (BSP): APIC ID:  0
>  cpu1 (AP): APIC ID:  1
> random: unblocking device.
> ioapic0 <Version 2.0> irqs 0-23 on motherboard
> ioapic1 <Version 2.0> irqs 24-47 on motherboard
> random: entropy device external interface
> kbd1 at kbdmux0
> netmap: loaded module
> module_register_init: MOD_LOAD (vesa, 0xffffffff80ee2ef0, 0) error 19
> vtvga0: <VT VGA driver> on motherboard
> cryptosoft0: <software crypto> on motherboard
> acpi0: <INTEL S5000PAL> on motherboard
> acpi0: Power Button (fixed)
> cpu0: <ACPI CPU> on acpi0
> cpu1: <ACPI CPU> on acpi0
> atrtc0: <AT realtime clock> port 0x70-0x71,0x74-0x77 irq 8 on acpi0
> Event timer "RTC" frequency 32768 Hz quality 0
> attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
> Timecounter "i8254" frequency 1193182 Hz quality 0
> Event timer "i8254" frequency 1193182 Hz quality 100
> hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
> Timecounter "HPET" frequency 14318180 Hz quality 950
> Event timer "HPET" frequency 14318180 Hz quality 450
> Event timer "HPET1" frequency 14318180 Hz quality 440
> Event timer "HPET2" frequency 14318180 Hz quality 440
> Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
> acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
> acpi_button0: <Power Button> on acpi0
> pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
> pci0: <ACPI PCI bus> on pcib0
> pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0
> pci1: <ACPI PCI bus> on pcib1
> pcib2: <ACPI PCI-PCI bridge> irq 16 at device 0.0 on pci1
> pci2: <ACPI PCI bus> on pcib2
> pcib3: <ACPI PCI-PCI bridge> irq 16 at device 0.0 on pci2
> pci3: <ACPI PCI bus> on pcib3
> pcib4: <ACPI PCI-PCI bridge> at device 0.0 on pci3
> pci4: <ACPI PCI bus> on pcib4
> mfi0: <LSI MegaSAS 1064R> mem 0xb8b00000-0xb8b0ffff,0xb8900000-0xb891ffff irq
> 18 at device 14.0 on pci4 mfi0: Using MSI
> mfi0: Megaraid SAS driver Ver 4.23 
> mfi0: FW MaxCmds = 1008, limiting to 128
> pcib5: <PCI-PCI bridge> at device 0.2 on pci3
> pci5: <PCI bus> on pcib5
> pcib6: <ACPI PCI-PCI bridge> irq 16 at device 1.0 on pci2
> pci6: <ACPI PCI bus> on pcib6
> pcib7: <ACPI PCI-PCI bridge> irq 16 at device 2.0 on pci2
> pci7: <ACPI PCI bus> on pcib7
> em0: <Intel(R) PRO/1000 Network Connection 7.6.1-k> port 0x3020-0x303f mem
> 0xb8820000-0xb883ffff,0xb8400000-0xb87fffff irq 18 at device 0.0 on pci7
> em0: Using an MSI interrupt
> em0: Ethernet address: 00:15:17:db:55:98
> em0: netmap queues/slots: TX 1/1024, RX 1/1024
> em1: <Intel(R) PRO/1000 Network Connection 7.6.1-k> port 0x3000-0x301f mem
> 0xb8800000-0xb881ffff,0xb8000000-0xb83fffff irq 19 at device 0.1 on pci7
> em1: Using an MSI interrupt
> em1: Ethernet address: 00:15:17:db:55:99
> em1: netmap queues/slots: TX 1/1024, RX 1/1024
> pcib8: <ACPI PCI-PCI bridge> at device 0.3 on pci1
> pci8: <ACPI PCI bus> on pcib8
> pcib9: <PCI-PCI bridge> at device 3.0 on pci0
> pci9: <PCI bus> on pcib9
> pcib10: <ACPI PCI-PCI bridge> at device 4.0 on pci0
> pci10: <ACPI PCI bus> on pcib10
> pcib11: <ACPI PCI-PCI bridge> at device 5.0 on pci0
> pci11: <ACPI PCI bus> on pcib11
> mps0: <Avago Technologies (LSI) SAS2308> port 0x2000-0x20ff mem
> 0xb8d40000-0xb8d4ffff,0xb8d00000-0xb8d3ffff irq 18 at device 0.0 on pci11
> mps0: Firmware: 20.00.02.00, Driver: 20.00.00.00-fbsd
> mps0: IOCCapabilities:
> 5285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>
> pcib12: <ACPI PCI-PCI bridge> at device 6.0 on pci0
> pci12: <ACPI PCI bus> on pcib12
> pcib13: <ACPI PCI-PCI bridge> at device 7.0 on pci0
> pci13: <ACPI PCI bus> on pcib13
> uhci0: <Intel 631XESB/632XESB/3100 USB controller USB-1> port 0x4080-0x409f
> irq 23 at device 29.0 on pci0 uhci0: LegSup = 0x2f00
> usbus0 on uhci0
> uhci1: <Intel 631XESB/632XESB/3100 USB controller USB-2> port 0x4060-0x407f
> irq 22 at device 29.1 on pci0 uhci1: LegSup = 0x2f00
> usbus1 on uhci1
> uhci2: <Intel 631XESB/632XESB/3100 USB controller USB-3> port 0x4040-0x405f
> irq 23 at device 29.2 on pci0 uhci2: LegSup = 0x2f00
> usbus2 on uhci2
> uhci3: <Intel 631XESB/632XESB/3100 USB controller USB-4> port 0x4020-0x403f
> irq 22 at device 29.3 on pci0 uhci3: LegSup = 0x2f00
> usbus3 on uhci3
> ehci0: <Intel 63XXESB USB 2.0 controller> mem 0xb8e00400-0xb8e007ff irq 23 at
> device 29.7 on pci0 usbus4: EHCI version 1.0
> usbus4 on ehci0
> pcib14: <ACPI PCI-PCI bridge> at device 30.0 on pci0
> pci14: <ACPI PCI bus> on pcib14
> vgapci0: <VGA-compatible display> port 0x1000-0x10ff mem
> 0xb0000000-0xb7ffffff,0xb8c00000-0xb8c0ffff irq 17 at device 12.0 on pci14
> vgapci0: Boot video device
> isab0: <PCI-ISA bridge> at device 31.0 on pci0
> isa0: <ISA bus> on isab0
> atapci0: <Intel 63XXESB2 UDMA100 controller> port
> 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x40b0-0x40bf irq 20 at device 31.1 on
> pci0 ata0: <ATA channel> at channel 0 on atapci0
> atapci1: <Intel 63XXESB2 SATA300 controller> port
> 0x40c8-0x40cf,0x40e4-0x40e7,0x40c0-0x40c7,0x40e0-0x40e3,0x40a0-0x40af mem
> 0xb8e00000-0xb8e003ff irq 20 at device 31.2 on pci0 ata2: <ATA channel> at
> channel 0 on atapci1 ata3: <ATA channel> at channel 1 on atapci1
> uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
> uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
> atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
> atkbd0: <AT Keyboard> irq 1 on atkbdc0
> kbd0 at atkbd0
> atkbd0: [GIANT-LOCKED]
> orm0: <ISA Option ROMs> at iomem
> 0xc0000-0xc8fff,0xce800-0xcf7ff,0xcf800-0xd07ff,0xd0800-0xd67ff on isa0
> ppc0: cannot reserve I/O port range
> est0: <Enhanced SpeedStep Frequency Control> on cpu0
> est1: <Enhanced SpeedStep Frequency Control> on cpu1
> mfi0: 19700 (509494462s/0x0020/info) - Shutdown command received from host
> mfi0: 19701 (boot + 0s/0x0020/info) - Firmware initialization started (PCI ID
> 0411/1000/3501/8086) mfi0: 19702 (boot + 0s/0x0020/info) - Firmware version
> 1.12.280-0826 mfi0: 19703 (boot + 23s/0x0008/info) - Battery temperature is
> normal mfi0: 19704 (boot + 23s/0x0008/info) - Battery Present
> mfi0: 19705 (boot + 24s/0x0020/info) - Board Revision 
> mfi0: 19706 (boot + 53s/0x0004/info) - Enclosure (SES) discovered on PD 07(c
> None/p1) mfi0: 19707 (boot + 53s/0x0002/info) - Inserted: Encl PD 07
> mfi0: MFI_DCMD_PD_LIST_QUERY failed 2
> mfi0: 19708 (boot + 53s/0x0002/info) - Inserted: PD 07(c None/p1) Info:
> enclPd=07, scsiType=d, portMap=09, sasAddr=5000007000580000,0000000000000000
> mfi0: 19709 (boot + 53s/0x0002/info) - Inserted: PD 1c(e0x07/s0)
> mfi0: MFI_DCMD_PD_LIST_QUERY failed 2
> mfi0: 19710 (boot + 53s/0x0002/info) - Inserted: PD 1c(e0x07/s0) Info:
> enclPd=07, scsiType=0, portMap=00, sasAddr=83945143a499a876,0000000000000000
> mfi0: 19711 (boot + 53s/0x0002/info) - Inserted: PD 1d(e0x07/s1)
> mfi0: MFI_DCMD_PD_LIST_QUERY failed 2
> mfi0: 19712 (boot + 53s/0x0002/info) - Inserted: PD 1d(e0x07/s1) Info:
> enclPd=07, scsiType=0, portMap=01, sasAddr=839544489e9aad93,0000000000000000
> mfi0: 19713 (boot + 53s/0x0002/info) - Inserted: PD 1e(e0x07/s2)
> mfi0: MFI_DCMD_PD_LIST_QUERY failed 2
> mfi0: 19714 (boot + 53s/0x0002/info) - Inserted: PD 1e(e0x07/s2) Info:
> enclPd=07, scsiType=0, portMap=02, sasAddr=83985540c098cb92,0000000000000000
> mfi0: 19715 (boot + 53s/0x0002/info) - Inserted: PD 1f(e0x07/s3)
> mfi0: MFI_DCMD_PD_LIST_QUERY failed 2
> mfi0: 19716 (boot + 53s/0x0002/info) - Inserted: PD 1f(e0x07/s3) Info:
> enclPd=07, scsiType=0, portMap=03, sasAddr=8394473ac3acd095,0000000000000000
> mfi0: 19717 (509496114s/0x0020/info) - Time established as 02/22/16 22:41:54;
> (55 seconds since power on) usbus0: 12Mbps Full Speed USB v1.0
> Timecounters tick every 1.000 msec
> IPsec: Initialized Security Association Processing.
> usbus1: 12Mbps Full Speed USB v1.0
> usbus2: 12Mbps Full Speed USB v1.0
> ugen0.1: <Intel> at usbus0
> ugen1.1: <Intel> at usbus1
> uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
> uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
> ugen2.1: <Intel> at usbus2
> uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
> usbus3: 12Mbps Full Speed USB v1.0
> usbus4: 480Mbps High Speed USB v2.0
> ugen3.1: <Intel> at usbus3
> ugen4.1: <Intel> at usbus4
> uhub3: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus3
> uhub4: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus4
> mfid0 on mfi0
> mfid0: 952720MB (1951170560 sectors) RAID volume (no label) is optimal
> uhub0: 2 ports with 2 removable, self powered
> uhub1: 2 ports with 2 removable, self powered
> uhub2: 2 ports with 2 removable, self powered
> mfid1 on mfi0
> mfid1: 952720MB (1951170560 sectors) RAID volume (no label) is optimal
> mps0: SAS Address for SATA device = 83ae5139c8babb8a
> uhub3: 2 ports with 2 removable, self powered
> mps0: SAS Address for SATA device = 83ac5340bec1a876
> mps0: SAS Address for SATA device = 83984f3cc698c081
> mps0: SAS Address for SATA device = 83b43946b6a1cc77
> mps0: SAS Address for SATA device = 83984b3ab4b2cd83
> mps0: SAS Address for SATA device = 839a5843b3baaf75
> mps0: SAS Address for SATA device = 839a4733c7add186
> mps0: SAS Address for SATA device = 83a95744a5b4c696
> mps0: SAS Address for SATA device = 7e9735279fa59487
> mps0: SAS Address for SATA device = 7e973622a2a7958c
> mps0: SAS Address from SATA device = 83ae5139c8babb8a
> mps0: SAS Address from SATA device = 83ac5340bec1a876
> mps0: SAS Address from SATA device = 83984f3cc698c081
> mps0: SAS Address from SATA device = 83b43946b6a1cc77
> mps0: SAS Address from SATA device = 83984b3ab4b2cd83
> mps0: SAS Address from SATA device = 839a5843b3baaf75
> mps0: SAS Address from SATA device = 839a4733c7add186
> mps0: SAS Address from SATA device = 83a95744a5b4c696
> mps0: SAS Address from SATA device = 7e9735279fa59487
> mps0: SAS Address from SATA device = 7e973622a2a7958c
> mps0: SAS Address for SATA device = 7e9737209fa99788
> mps0: SAS Address for SATA device = 7e973321a7ad958c
> mps0: SAS Address from SATA device = 7e9737209fa99788
> mps0: SAS Address from SATA device = 7e973321a7ad958c
> ses0 at mps0 bus 0 scbus0 target 20 lun 0
> ses0: <LSI SAS2X28 0e12> Fixed Enclosure Services SPC-3 SCSI device
> ses0: Serial Number 
> ses0: 600.000MB/s transfers
> ses0: Command Queueing enabled
> ses0: SCSI-3 ENC Device
> ses1 at mps0 bus 0 scbus0 target 21 lun 0
> ses1: <LSI SAS2X28 0e12> Fixed Enclosure Services SPC-3 SCSI device
> ses1: Serial Number 
> ses1: 600.000MB/s transfers
> ses1: Command Queueing enabled
> ses1: SCSI-3 ENC Device
> da0 at mps0 bus 0 scbus0 target 8 lun 0
> da0: <ATA WDC WD1003FZEX-0 1A01> Fixed Direct Access SPC-4 SCSI device
> da0: Serial Number      WD-WCC3FJNKZRCJ
> da0: 600.000MB/s transfers
> da0: Command Queueing enabled
> da0: 953869MB (1953525168 512 byte sectors)
> da1 at mps0 bus 0 scbus0 target 9 lun 0
> da1: <ATA WDC WD1003FZEX-0 1A01> Fixed Direct Access SPC-4 SCSI device
> da1: Serial Number      WD-WCC3FHPRPY06
> da1: 600.000MB/s transfers
> da1: Command Queueing enabled
> da1: 953869MB (1953525168 512 byte sectors)
> da2 at mps0 bus 0 scbus0 target 10 lun 0
> da2: <ATA WDC WD1003FZEX-0 1A01> Fixed Direct Access SPC-4 SCSI device
> da2: Serial Number      WD-WCC3F4LNX0HA
> da2: 600.000MB/s transfers
> da2: Command Queueing enabled
> da2: 953869MB (1953525168 512 byte sectors)
> da3 at mps0 bus 0 scbus0 target 11 lun 0
> da3: <ATA WDC WD1003FZEX-0 1A01> Fixed Direct Access SPC-4 SCSI device
> da3: Serial Number      WD-WCC3FP6XH9T7
> da3: 600.000MB/s transfers
> da3: Command Queueing enabled
> da3: 953869MB (1953525168 512 byte sectors)
> da4 at mps0 bus 0 scbus0 target 12 lun 0
> da4: <ATA WDC WD1003FZEX-0 1A01> Fixed Direct Access SPC-4 SCSI device
> da4: Serial Number      WD-WCC3F4HLFJUC
> da4: 600.000MB/s transfers
> da4: Command Queueing enabled
> da4: 953869MB (1953525168 512 byte sectors)
> da5 at mps0 bus 0 scbus0 target 13 lun 0
> da5: <ATA WDC WD1003FZEX-0 1A01> Fixed Direct Access SPC-4 SCSI device
> da5: Serial Number      WD-WCC3F6UUER75
> da5: 600.000MB/s transfers
> da5: Command Queueing enabled
> da5: 953869MB (1953525168 512 byte sectors)
> da6 at mps0 bus 0 scbus0 target 14 lun 0
> da6: <ATA WDC WD1003FZEX-0 1A01> Fixed Direct Access SPC-4 SCSI device
> da6: Serial Number      WD-WCC3F6DEYEYF
> da6: 600.000MB/s transfers
> da6: Command Queueing enabled
> da6: 953869MB (1953525168 512 byte sectors)
> da7 at mps0 bus 0 scbus0 target 15 lun 0
> da7: <ATA WDC WD1003FZEX-0 1A01> Fixed Direct Access SPC-4 SCSI device
> da7: Serial Number      WD-WCC3FETV7LNV
> da7: 600.000MB/s transfers
> da7: Command Queueing enabled
> da7: 953869MB (1953525168 512 byte sectors)
> da9 at mps0 bus 0 scbus0 target 17 lun 0
> da9: <ATA WDC WD1002FAEX-0 1D05> Fixed Direct Access SPC-4 SCSI device
> da9: Serial Number      WD-WCATRC344375
> da9: 600.000MB/s transfers
> da9: Command Queueing enabled
> da9: 953869MB (1953525168 512 byte sectors)
> da8 at mps0 bus 0 scbus0 target 16 lun 0
> ses0: da0,pass0: Element descriptor: 'Slot 01'
> da8: <ATA WDC WD1002FAEX-0 1D05> Fixed Direct Access SPC-4 SCSI device
> ses0: da0,pass0: SAS Device Slot Element: 1 Phys at Slot 0
> da8: Serial Number      WD-WCATRC291160
> ses0:  phy 0: SATA device
> da8: 600.000MB/s transfersses0:  phy 0: parent 5003048001a23a3f addr
> 5003048001a23a2c 
>
> da8: Command Queueing enabled
> da8: 953869MB (1953525168 512 byte sectors)
> da10 at mps0 bus 0 scbus0 target 18 lun 0
> da10: <ATA WDC WD1002FAEX-0 1D05> Fixed Direct Access SPC-4 SCSI device
> da10: Serial Number      WD-WCATRC421591
> da10: 600.000MB/s transfers
> da10: Command Queueing enabled
> da10: 953869MB (1953525168 512 byte sectors)
> da11 at mps0 bus 0 scbus0 target 19 lun 0
> da11: <ATA WDC WD1002FAEX-0 1D05> Fixed Direct Access SPC-4 SCSI device
> da11: Serial Number      WD-WCATRC039975
> da11: 600.000MB/s transfers
> da11: Command Queueing enabled
> da11: 953869MB (1953525168 512 byte sectors)
> ses0: da1,pass1: Element descriptor: 'Slot 02'
> ses0: da1,pass1: SAS Device Slot Element: 1 Phys at Slot 1
> ses0:  phy 0: SATA device
> ses0:  phy 0: parent 5003048001a23a3f addr 5003048001a23a2d
> ses0: da2,pass2: Element descriptor: 'Slot 03'
> ses0: da2,pass2: SAS Device Slot Element: 1 Phys at Slot 2
> ses0:  phy 0: SATA device
> ses0:  phy 0: parent 5003048001a23a3f addr 5003048001a23a2e
> ses0: da3: Element descriptor: 'Slot 04'
> ses0: da3: SAS Device Slot Element: 1 Phys at Slot 3
> ses0:  phy 0: SATA device
> ses0:  phy 0: parent 5003048001a23a3f addr 5003048001a23a2f
> ses0: da4: Element descriptor: 'Slot 05'
> ses0: da4: SAS Device Slot Element: 1 Phys at Slot 4
> ses0:  phy 0: SATA device
> ses0:  phy 0: parent 5003048001a23a3f addr 5003048001a23a30
> ses0: da5: Element descriptor: 'Slot 06'
> ses0: da5: SAS Device Slot Element: 1 Phys at Slot 5
> ses0:  phy 0: SATA device
> ses0:  phy 0: parent 5003048001a23a3f addr 5003048001a23a31
> ses0: da6: Element descriptor: 'Slot 07'
> ses0: da6: SAS Device Slot Element: 1 Phys at Slot 6
> ses0:  phy 0: SATA device
> ses0:  phy 0: parent 5003048001a23a3f addr 5003048001a23a32
> ses0: da7: Element descriptor: 'Slot 08'
> ses0: da7: SAS Device Slot Element: 1 Phys at Slot 7
> ses0:  phy 0: SATA device
> ses0:  phy 0: parent 5003048001a23a3f addr 5003048001a23a33
> ses0: da8: Element descriptor: 'Slot 09'
> ses0: da8: SAS Device Slot Element: 1 Phys at Slot 8
> ses0:  phy 0: SATA device
> ses0:  phy 0: parent 5003048001a23a3f addr 5003048001a23a34
> ses0: da9: Element descriptor: 'Slot 10'
> ses0: da9: SAS Device Slot Element: 1 Phys at Slot 9
> ses0:  phy 0: SATA device
> ses0:  phy 0: parent 5003048001a23a3f addr 5003048001a23a35
> ses0: da10: Element descriptor: 'Slot 11'
> ses0: da10: SAS Device Slot Element: 1 Phys at Slot 10
> ses0:  phy 0: SATA device
> ses0:  phy 0: parent 5003048001a23a3f addr 5003048001a23a36
> ses0: da11: Element descriptor: 'Slot 12'
> ses0: da11: SAS Device Slot Element: 1 Phys at Slot 11
> ses0:  phy 0: SATA device
> ses0:  phy 0: parent 5003048001a23a3f addr 5003048001a23a37
> cd0 at ata0 bus 0 scbus1 target 1 lun 0
> cd0: <MATSHITA DVD-ROM UJDA780 1.50> Removable CD-ROM SCSI device
> cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes)
> cd0: Attempt to query device size failed: NOT READY, Medium not present -
> tray closed SMP: AP CPU #1 Launched!
> Timecounter "TSC-low" frequency 1330028520 Hz quality 1000
> WARNING: WITNESS option enabled, expect reduced performance.
> Trying to mount root from ufs:/dev/mfid0p2 [rw]...
> ZFS filesystem version: 5
> ZFS storage pool version: features support (5000)
> uhub4: 8 ports with 8 removable, self powered
> ugen2.2: <No brand> at usbus2
> ukbd0: <No brand KVM, class 0/0, rev 1.10/0.00, addr 2> on usbus2
> kbd2 at ukbd0
> ugen4.2: <IntelR> at usbus4
> umass0: <IntelR Multidevice, class 0/0, rev 2.00/0.01, addr 2> on usbus4
> umass0:  SCSI over Bulk-Only; quirks = 0x0100
> umass0:4:0: Attached to scbus4
> umass1: <IntelR Multidevice, class 0/0, rev 2.00/0.01, addr 2> on usbus4
> umass1:  SCSI over Bulk-Only; quirks = 0x0100
> umass1:5:1: Attached to scbus5
> umass2: <IntelR Multidevice, class 0/0, rev 2.00/0.01, addr 2> on usbus4
> umass2:  SCSI over Bulk-Only; quirks = 0x0100
> umass2:6:2: Attached to scbus6
> umass3: <IntelR Multidevice, class 0/0, rev 2.00/0.01, addr 2> on usbus4
> umass3:  SCSI over Bulk-Only; quirks = 0x0100
> umass3:7:3: Attached to scbus7
> ukbd1: <IntelR Multidevice, class 0/0, rev 2.00/0.01, addr 2> on usbus4
> kbd3 at ukbd1
> cd1 at umass-sim2 bus 2 scbus6 target 0 lun 0
> cd1: <Intel(R) RMM2 VDrive 3 0.01> Removable CD-ROM SCSI-3 device
> cd1: Serial Number 731284497BE433D913F08AD68B5E9D71
> cd1: 40.000MB/s transfers
> cd1: Attempt to query device size failed: NOT READY, Medium not present
> cd1: quirks=0x10<10_BYTE_ONLY>
> cd2 at umass-sim3 bus 3 scbus7 target 0 lun 0
> cd2: <Intel(R) RMM2 VDrive 4 0.01> Removable CD-ROM SCSI-3 device
> cd2: Serial Number 731284497BE433D913F08AD68B5E9D71
> cd2: 40.000MB/s transfers
> cd2: Attempt to query device size failed: NOT READY, Medium not present
> cd2: quirks=0x10<10_BYTE_ONLY>
> cd3 at umass-sim0 bus 0 scbus4 target 0 lun 0
> cd3: <Intel(R) RMM2 VDrive 1 0.01> Removable CD-ROM SCSI-3 device
> cd3: Serial Number 731284497BE433D913F08AD68B5E9D71
> cd3: 40.000MB/s transfers
> cd3: Attempt to query device size failed: NOT READY, Medium not present
> cd3: quirks=0x10<10_BYTE_ONLY>
> cd4 at umass-sim1 bus 1 scbus5 target 0 lun 0
> cd4: <Intel(R) RMM2 VDrive 2 0.01> Removable CD-ROM SCSI-3 device
> cd4: Serial Number 731284497BE433D913F08AD68B5E9D71
> cd4: 40.000MB/s transfers
> cd4: Attempt to query device size failed: NOT READY, Medium not present
> cd4: quirks=0x10<10_BYTE_ONLY>
> em0: link state changed to UP
> uhid0: <IntelR Multidevice, class 0/0, rev 2.00/0.01, addr 2> on usbus4
> ums0: <No brand KVM, class 0/0, rev 1.10/0.00, addr 2> on usbus2
> ums0: 5 buttons and [XYZ] coordinates ID=1
> 
> 
> 
> 
> 
> -- 
> Dave Robison