[long post with long lines, sorry] I have the following old hardware which I'm trying to make into a storage server (back story elided). Tyan Thunder K8WE with dual Opteron 270 8GB REG ECC RAM 3ware/AMCC 9550SXU-16 SATA controller Adaptec 29160 SCSI card -> Quantum LTO3 tape ChenBro case and backplanes. 'don't remember' PSU. I do remember paying £98 3 years ago, so not cheap! floppy Some Seagate Barracuda drives. Two old 500GB for the O/S and 14 new 1.5TB for data (plus some spares). Astute readers will know that the 1.5TB units have a chequered history. I went to considerable effort to avoid being stuck with a bricked unit, so imagine my dismay when, just before I was about to post this, I discovered there's a new issue with these drives where they reallocate sectors, from new. I don't want to get sucked into a discussion about whether these disks are faulty or not. I want to examine what seems to be a regression between 7.2-RELEASE and 8-CURRENT. If you can't resist, start a thread in chat and CC me. Anyway, here's the full story (from memory I'm afraid). All disks exported as single drives (no JBOD anymore). Install current snapshot on da0 and gmirror with da1, both 500GB disks. Create a pool with the 14 1.5TB disks. Raidz2. Create a dataset. Tried various settings while benchmarking. Currently using compression=on copies=2. Copy ~1TB of data. Large files, dumps from the backup server. [50MB/s for 8-10h.] zpool scrub At some point (40m?) checksum errors start along with dire warnings about the card. Break the gmirror and install 7.2-RELEASE on da1. Delete the pool. Create an identical pool under 7.2 (version 6) and then scrub. Works okay, no errors. Export the pool, boot 8, import pool. Copy the data again. Scrub again. Same checksum throwing result. Cancel scrub, export pool, reboot 7.2, import pool. Scrubs okay. So it looks like I can scrub okay on 7.2, but not on 8. Data itself would seem to be okay. [I've shortened this a bit. For example at one point the 7.2-RELEASE stopped booting, had to reinstall. LORs come and go! Lagg has been in and out (I'm trying to benchmark the network as well). I did see 4 checksum errors on 7.2 but that was after 8 had 'fixed' the pool and was interrupted doing so. There were no card errors. Also subject to my memory. Must learn to write stuff down.] I also thought I'd try stressing the hardware without zfs envolved in case there was an issue with the drives/card/PCI bus or PSU. foreach x ( `jot -s " " -w "da%d" 14 2` ) dd if=/dev/$x of=/dev/null bs=512k& end It's crude and the load across the drives is uneven but it pulls ~45MB/s off the 'slowest' drive. Ran this for an hour, no errors. Note to self: double check you did this under current not 7.2. Better tests welcome. If it's three lines or fewer that's a bonus. At this point I install smartmontools and find reallocated sectors. Armed with this keyword Google gives me lots of hits :(( So I started again, this time checking for reallocated sectors each time. The numbers aren't going up, scrubs are still failing. Here's a sample of the messages. Scrub cancelled as soon as messages start. Jun 12 20:02:00 data kernel: lock order reversal: Jun 12 20:02:00 data kernel: 1st 0xffffff0004e00448 ufs (ufs) _at_ /usr/src.8/sys/kern/vfs_mount.c:1197 Jun 12 20:02:00 data kernel: 2nd 0xffffff0004dd5d80 devfs (devfs) _at_ /usr/src.8/sys/ufs/ffs/ffs_vfsops.c:1194 Jun 12 20:02:00 data kernel: KDB: stack backtrace: Jun 12 20:02:00 data kernel: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a Jun 12 20:02:00 data kernel: _witness_debugger() at _witness_debugger+0x2e Jun 12 20:02:00 data kernel: witness_checkorder() at witness_checkorder+0x81e Jun 12 20:02:00 data kernel: __lockmgr_args() at __lockmgr_args+0xc2a Jun 12 20:02:00 data kernel: vop_stdlock() at vop_stdlock+0x39 Jun 12 20:02:00 data kernel: VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b Jun 12 20:02:00 data kernel: _vn_lock() at _vn_lock+0x47 Jun 12 20:02:00 data kernel: ffs_flushfiles() at ffs_flushfiles+0xb5 Jun 12 20:02:00 data kernel: ffs_unmount() at ffs_unmount+0x67 Jun 12 20:02:00 data kernel: dounmount() at dounmount+0x2e6 Jun 12 20:02:00 data kernel: unmount() at unmount+0x250 Jun 12 20:02:00 data kernel: syscall() at syscall+0x1af Jun 12 20:02:00 data kernel: Xfast_syscall() at Xfast_syscall+0xd0 Jun 12 20:02:00 data kernel: --- syscall (22, FreeBSD ELF64, unmount), rip = 0x8006a049c, rsp = 0x7fffffffe308, rbp = 0 --- Jun 12 20:19:30 data kernel: This module (opensolaris) contains code covered by the Jun 12 20:19:30 data kernel: Common Development and Distribution License (CDDL) Jun 12 20:19:30 data kernel: see http://opensolaris.org/os/licensing/opensolaris_license/ Jun 12 20:19:30 data kernel: WARNING: ZFS is considered to be an experimental feature in FreeBSD. Jun 12 20:19:30 data kernel: ZFS filesystem version 13 Jun 12 20:19:30 data kernel: ZFS storage pool version 13 Jun 12 20:46:42 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da10 offset=8845411328 size=10752 Jun 12 20:46:42 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da8 offset=8845422080 size=11264 Jun 12 20:46:42 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da5 offset=8845433344 size=11264 Jun 12 20:46:42 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da13 offset=8845444096 size=10752 Jun 12 20:46:42 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da7 offset=8845455360 size=10752 Jun 12 20:46:43 data kernel: twa0: ERROR: (0x16: 0x1301): Missing expected status bit(s): status reg = 0xde0d530c; Missing bits: [MC_RDY,] Jun 12 20:46:43 data kernel: twa0: ERROR: (0x03: 0x0107): Duplicate request ID: RequestID=188 Jun 12 20:46:43 data kernel: (da3:twa0:0:3:0): READ(10). CDB: 28 0 21 ba 71 8c 0 0 2b 0 Jun 12 20:46:43 data kernel: (da3:twa0:0:3:0): CAM Status: SCSI Status Error Jun 12 20:46:43 data kernel: (da3:twa0:0:3:0): SCSI Status: Check Condition Jun 12 20:46:43 data kernel: (da3:twa0:0:3:0): ILLEGAL REQUEST asc:24,0 Jun 12 20:46:43 data kernel: (da3:twa0:0:3:0): Invalid field in CDB Jun 12 20:46:43 data kernel: (da3:twa0:0:3:0): Unretryable error Jun 12 20:46:43 data kernel: twa0: ERROR: (0x15: 0x1201): Unposted command completed!!: request = 0xffffff80003f4ac4, status = 3 Jun 12 20:46:43 data kernel: twa0: INFO: (0x16: 0x1108): Resetting controller...: Jun 12 20:46:43 data kernel: twa0: INFO: (0x04: 0x005E): Cache synchronization completed: unit=0 Jun 12 20:46:43 data kernel: twa0: INFO: (0x04: 0x0001): Controller reset occurred: resets=1 Jun 12 20:46:43 data kernel: twa0: INFO: (0x16: 0x1107): Controller reset done!: Jun 12 20:46:43 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da8 offset=8849261056 size=10752 Jun 12 20:46:43 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da7 offset=289718205952 size=11264 Jun 12 20:46:43 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da4 offset=8850237440 size=11264 Jun 12 20:46:43 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da10 offset=8850237440 size=10752 Jun 12 20:46:43 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da8 offset=289718480384 size=10752 Jun 12 20:46:43 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da15 offset=289718831104 size=11264 Jun 12 20:46:43 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da2 offset=289718908416 size=10752 Jun 12 20:46:43 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da6 offset=289718282752 size=11264 Jun 12 20:46:43 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da9 offset=289718282752 size=11264 Jun 12 20:46:43 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da3 offset=289718875648 size=10752 Jun 12 20:46:43 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da2 offset=8849897472 size=11264 Jun 12 20:46:43 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da9 offset=8849897472 size=10752 Jun 12 20:46:43 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da12 offset=289718557184 size=10752 Jun 12 20:46:43 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da15 offset=289719237120 size=10752 Jun 12 20:46:43 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da15 offset=8850511360 size=11264 Jun 12 20:46:43 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da14 offset=289719423488 size=11264 Jun 12 20:46:43 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da5 offset=8849952256 size=11264 Jun 12 20:46:43 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da10 offset=8849952256 size=10752 Jun 12 20:46:43 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da6 offset=289718579200 size=10752 Jun 12 20:46:43 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da10 offset=289718579200 size=10752 Jun 12 20:46:47 data root: ZFS: vdev I/O failure, zpool=tank path=/dev/da3 offset=289723783168 size=87552 error=22 Jun 12 20:46:47 data root: ZFS: vdev I/O failure, zpool=tank path=/dev/da3 offset=289723783168 size=10752 error=22 Jun 12 20:46:47 data root: ZFS: vdev I/O failure, zpool=tank path=/dev/da3 offset=289723793920 size=10752 error=22 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da13 offset=8851531776 size=11264 Jun 12 20:46:47 data root: ZFS: vdev I/O failure, zpool=tank path=/dev/da3 offset=289723804672 size=11264 error=22 Jun 12 20:46:47 data root: ZFS: vdev I/O failure, zpool=tank path=/dev/da3 offset=289723815936 size=10752 error=22 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da10 offset=8851312640 size=10752 Jun 12 20:46:47 data root: ZFS: vdev I/O failure, zpool=tank path=/dev/da3 offset=289723826688 size=11264 error=22 Jun 12 20:46:47 data root: ZFS: vdev I/O failure, zpool=tank path=/dev/da3 offset=289723837952 size=10752 error=22 Jun 12 20:46:47 data root: ZFS: vdev I/O failure, zpool=tank path=/dev/da3 offset=289723848704 size=11264 error=22 Jun 12 20:46:47 data root: ZFS: vdev I/O failure, zpool=tank path=/dev/da3 offset=289723859968 size=10752 error=22 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da13 offset=8851543040 size=10752 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da10 offset=8851454976 size=11264 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da8 offset=8851565056 size=10752 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da2 offset=8851235840 size=11264 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da6 offset=8851575808 size=11264 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da13 offset=8851499008 size=10752 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da15 offset=8852255744 size=11264 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da11 offset=8851971072 size=10752 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da11 offset=8851718656 size=10752 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da12 offset=289720421888 size=11264 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da4 offset=289720817152 size=11264 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da8 offset=289720948736 size=11264 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da11 offset=289721003520 size=11264 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da8 offset=289720718336 size=11264 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da6 offset=289721212416 size=10752 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da8 offset=289721332736 size=11264 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da8 offset=289721344000 size=10752 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da14 offset=289720443904 size=11264 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da14 offset=289721409536 size=10752 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da14 offset=289721420288 size=11264 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da7 offset=289720509952 size=11264 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da12 offset=8851795456 size=10752 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da13 offset=8851795456 size=10752 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da12 offset=8851806208 size=10752 Jun 12 20:46:47 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da12 offset=8851850240 size=10752 Jun 12 20:46:48 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da3 offset=573848415232 size=10752 Jun 12 20:46:48 data kernel: twa0: ERROR: (0x16: 0x1301): Missing expected status bit(s): status reg = 0x7f9bdff5; Missing bits: [MC_RDY,] Jun 12 20:46:48 data kernel: twa0: ERROR: (0x16: 0x1302): Unexpected status bit(s): status reg = 0x900000 Unexpected bits: [PCI_ABRT,PCI_PERR] Jun 12 20:46:48 data kernel: twa0: _PERR]: (0x4943502C: 0x1303): PCI parity error: clearing... Re-seat/move/replace card: status reg = 0x7f9bdff5 [CMD_Q_EMPTY,RESP_Q_EMPTY,CMD_Q_FULL,RESP_INTR,CMD_INTR,HOST_INTR,PCI_ABRT,PCI_PERR] Jun 12 20:46:48 data kernel: twa0: _PERR]: (0x4943502C: 0x1304): PCI abort: clearing... : status reg = 0x7f9bdff5 [CMD_Q_EMPTY,RESP_Q_EMPTY,CMD_Q_FULL,RESP_INTR,CMD_INTR,HOST_INTR,PCI_ABRT,PCI_PERR] Jun 12 20:46:48 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da11 offset=573849005056 size=11264 Jun 12 20:46:48 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da14 offset=573848699904 size=11264 Jun 12 20:46:52 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da5 offset=289734832640 size=10752 Jun 12 20:46:52 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da5 offset=289734843392 size=10752 Jun 12 20:46:52 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da14 offset=289734963712 size=10752 Jun 12 20:46:52 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da10 offset=289735402496 size=11264 Jun 12 20:46:52 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da13 offset=9262030848 size=10752 Jun 12 20:46:52 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da12 offset=9262228480 size=10752 Jun 12 20:46:52 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da7 offset=9262502912 size=10752 Jun 12 20:46:52 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da9 offset=9262469632 size=11264 Jun 12 20:46:52 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da8 offset=289735830528 size=11264 Jun 12 20:46:52 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da11 offset=289735874560 size=10752 Jun 12 20:46:52 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da6 offset=289736028160 size=11264 Jun 12 20:46:52 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da10 offset=289735731712 size=11264 Jun 12 20:46:52 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da2 offset=289735479808 size=10752 Jun 12 20:46:52 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da8 offset=289735479808 size=10752 Jun 12 20:46:52 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da13 offset=289735589376 size=10752 Jun 12 20:46:52 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da15 offset=289735512064 size=11264 Jun 12 20:46:52 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da11 offset=289735523328 size=11264 Jun 12 20:46:52 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da7 offset=9267793408 size=10752 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da11 offset=9267540992 size=10752 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da4 offset=9267573760 size=11264 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da7 offset=9267836928 size=11264 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da4 offset=9267595776 size=11264 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da6 offset=9268056576 size=10752 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da6 offset=9267848192 size=10752 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da11 offset=9267474944 size=10752 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da11 offset=9267485696 size=11264 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da12 offset=9268122112 size=11264 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da7 offset=9267913728 size=11264 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da6 offset=9267935744 size=11264 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da4 offset=9267607040 size=10752 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da12 offset=9268111360 size=10752 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da2 offset=289741294592 size=11264 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da7 offset=289741185024 size=10752 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da7 offset=289741195776 size=10752 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da8 offset=9267990528 size=11264 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da9 offset=289741228544 size=11264 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da10 offset=289741272576 size=10752 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da10 offset=289741283328 size=11264 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da12 offset=289741634560 size=10752 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da15 offset=289741864960 size=10752 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da15 offset=289741875712 size=10752 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da3 offset=289741755392 size=11264 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da3 offset=289740998656 size=10752 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da11 offset=289741008896 size=11264 Jun 12 20:46:53 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da2 offset=9268254208 size=10752 Jun 12 20:46:57 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da15 offset=9396110848 size=10752 Jun 12 20:46:57 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da5 offset=9397246464 size=10752 Jun 12 20:46:57 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da10 offset=573881940992 size=11264 Jun 12 20:46:57 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da6 offset=573881985024 size=11264 Jun 12 20:46:57 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da9 offset=573882555392 size=11264 Jun 12 20:46:57 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da6 offset=573882007040 size=10752 Jun 12 20:46:57 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da8 offset=573882007040 size=10752 Jun 12 20:46:57 data kernel: twa0: ERROR: (0x16: 0x1302): Unexpected status bit(s): status reg = 0x500000 Unexpected bits: [PCI_ABRT,Q_ERR,] Jun 12 20:46:57 data kernel: twa0: . : (0x2E2E676E: 0x1304): PCI abort: clearing... : status reg = 0x1c536664 [MC_RDY,RESP_Q_EMPTY,RESP_INTR,CMD_INTR,PCI_ABRT,Q_ERR,] Jun 12 20:46:57 data kernel: twa0: r: clearing... : (0x6F727265: 0x1305): Controller queue error: clearing... : status reg = 0x1c536664 [MC_RDY,RESP_Q_EMPTY,RESP_INTR,CMD_INTR,PCI_ABRT,Q_ERR,] Jun 12 20:46:57 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da2 offset=573884749824 size=11264 Jun 12 20:46:57 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da14 offset=573884058624 size=10752 Jun 12 20:46:57 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da2 offset=573884761088 size=10752 Jun 12 20:46:57 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da6 offset=573885046272 size=10752 Jun 12 20:46:59 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da3 offset=289755986432 size=10752 Jun 12 20:46:59 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da5 offset=289756029952 size=11264 Jun 12 20:46:59 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da9 offset=9545516544 size=11264 Jun 12 20:46:59 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da7 offset=289756062720 size=11264 Jun 12 20:47:00 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da12 offset=289756117504 size=11264 Jun 12 20:47:01 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da10 offset=289764136448 size=10752 Jun 12 20:47:01 data kernel: twa0: ERROR: (0x16: 0x1301): Missing expected status bit(s): status reg = 0x145994eb; Missing bits: [MC_RDY,] Jun 12 20:47:01 data kernel: twa0: ERROR: (0x16: 0x1302): Unexpected status bit(s): status reg = 0x500000 Unexpected bits: [PCI_ABRT,Q_ERR,] Jun 12 20:47:01 data kernel: twa0: ng... : (0x69726165: 0x1304): PCI abort: clearing... : status reg = 0x145994eb [CMD_Q_EMPTY,CMD_Q_FULL,RESP_INTR,HOST_INTR,PCI_ABRT,Q_ERR,] Jun 12 20:47:01 data kernel: twa0: error: clearing... : (0x20657565: 0x1305): Controller queue error: clearing... : status reg = 0x145994eb [CMD_Q_EMPTY,CMD_Q_FULL,RESP_INTR,HOST_INTR,PCI_ABRT,Q_ERR,] Jun 12 20:47:01 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da11 offset=9553024000 size=11264 Jun 12 20:47:01 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da12 offset=9553254400 size=11264 Jun 12 20:47:01 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da15 offset=9553112064 size=10752 Jun 12 20:47:01 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da8 offset=289764279296 size=10752 Jun 12 20:47:04 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da14 offset=289775233536 size=10752 Jun 12 20:47:04 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da6 offset=289775902720 size=11264 Jun 12 20:47:05 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da6 offset=9707735040 size=10752 Jun 12 20:47:05 data kernel: twa0: ERROR: (0x16: 0x1302): Unexpected status bit(s): status reg = 0x600000 Unexpected bits: [MC_ERR,Q_ERR,] Jun 12 20:47:05 data kernel: twa0: : (0x202E2E2E: 0x1305): Controller queue error: clearing... : status reg = 0x576a2c2e [MC_RDY,CMD_INTR,HOST_INTR,MC_ERR,Q_ERR,] Jun 12 20:47:05 data kernel: twa0: ERROR: (0x16: 0x1307): Micro-controller error! : status reg = 0x576a2c2e [MC_RDY,CMD_INTR,HOST_INTR,MC_ERR,Q_ERR,] Jun 12 20:47:05 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da7 offset=9708688384 size=10752 Jun 12 20:47:05 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da4 offset=289779238400 size=10752 Jun 12 20:47:07 data kernel: twa0: ERROR: (0x16: 0x1301): Missing expected status bit(s): status reg = 0x79c41042; Missing bits: [MC_RDY,] Jun 12 20:47:07 data kernel: twa0: ERROR: (0x16: 0x1302): Unexpected status bit(s): status reg = 0xc00000 Unexpected bits: [Q_ERR,PCI_PERR] Jun 12 20:47:07 data kernel: twa0: ove/replace card: (0x6D2F7461: 0x1303): PCI parity error: clearing... Re-seat/move/replace card: status reg = 0x79c41042 [CMD_Q_EMPTY,ATTN_INTR,Q_ERR,PCI_PERR] Jun 12 20:47:07 data kernel: twa0: ERROR: (0x20: 0x1305): Controller queue error: clearing... : status reg = 0x79c41042 [CMD_Q_EMPTY,ATTN_INTR,Q_ERR,PCI_PERR] Jun 12 20:47:07 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da13 offset=206088704 size=11264 Jun 12 20:47:07 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da13 offset=206099968 size=10752 Jun 12 20:47:07 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da10 offset=206396416 size=10752 Jun 12 20:47:07 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da11 offset=573904253952 size=10752 Jun 12 20:47:07 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da5 offset=573904166400 size=10752 Jun 12 20:47:07 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da5 offset=573904177152 size=11264 Jun 12 20:47:07 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da3 offset=573904681984 size=11264 Jun 12 20:47:07 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da3 offset=573904693248 size=10752 Jun 12 20:47:07 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da9 offset=573905043968 size=10752 Jun 12 20:47:08 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da6 offset=289786880 size=11264 Jun 12 20:47:08 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da7 offset=289611264 size=11264 Jun 12 20:47:08 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da14 offset=289830400 size=11264 Jun 12 20:47:08 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da4 offset=289633280 size=11264 Jun 12 20:47:08 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da13 offset=289918464 size=10752 Jun 12 20:47:08 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da15 offset=289929216 size=11264 Jun 12 20:47:08 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da8 offset=573911089152 size=10752 Jun 12 20:47:08 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da14 offset=289841664 size=10752 Jun 12 20:47:08 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da8 offset=290094080 size=10752 Jun 12 20:47:08 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da8 offset=290104832 size=11264 Jun 12 20:47:08 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da2 offset=573911187968 size=11264 Jun 12 20:47:08 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da2 offset=573911440384 size=11264 Jun 12 20:47:08 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da4 offset=573910979584 size=10752 Jun 12 20:47:08 data root: ZFS: checksum mismatch, zpool=tank path=/dev/da9 offset=573911330304 size=11264 Cheers -- ian j hartReceived on Sat Jun 13 2009 - 20:11:24 UTC
This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:39:49 UTC