Re: ZFS panic under extreme circumstances (2/3 disks corrupted)

On May 25, 2009, at 06:19 PM, Freddie Cash wrote:

> On Mon, May 25, 2009 at 9:12 AM, Thomas Backman  
> <serenity_at_exscape.org> wrote:
>> On May 25, 2009, at 05:39 PM, Freddie Cash wrote:
>> Was this on FreeBSD?
>
> 64-bit FreeBSD 7.1 using ZFS v6.  SATA drives connected to 3Ware RAID
> controllers, but configured as "Single Drive" arrays not using
> hardware RAID in any way.
>
>> I have another unfortunate thing to note regarding this: after a  
>> reboot,
>> it's even impossible to tell *which disk* has gone bad, even if the  
>> pool is
>> "uncleared" but otherwise "healed". It simply says that a device  
>> has failed,
>> with no clue as to which one, since they're all "ONLINE"!
>
> Even when using -v?  zpool status -v

Yup, unfortunately.

[root_at_clone ~]# zpool status -v test
   pool: test
  state: ONLINE
status: One or more devices has experienced an error resulting in data
	corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
	entire pool from backup.
    see: http://www.sun.com/msg/ZFS-8000-8A
  scrub: scrub completed after 0h0m with 372 errors on Mon May 25  
12:57:46 2009
config:

	NAME        STATE     READ WRITE CKSUM
	test        ONLINE       0     0   373
	  raidz1    ONLINE       0     0   747
	    da1     ONLINE       0     0     0  9K repaired
	    da2     ONLINE       0     0   171  119K repaired
	    da3     ONLINE       0     0     0  9K repaired
	  raidz1    ONLINE       0     0     2
	    ad0     ONLINE       0     0     0
	    ad1     ONLINE       0     0   315
	    ad3     ONLINE       0     0   318

errors: Permanent errors have been detected in the following files:

         /test/testfile
[root_at_clone ~]# reboot

[root_at_clone ~]# zpool status -xv
   pool: test
  state: ONLINE
status: One or more devices has experienced an error resulting in data
	corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
	entire pool from backup.
    see: http://www.sun.com/msg/ZFS-8000-8A
  scrub: none requested
config:

	NAME        STATE     READ WRITE CKSUM
	test        ONLINE       0     0     0
	  raidz1    ONLINE       0     0     0
	    da1     ONLINE       0     0     0
	    da2     ONLINE       0     0     0
	    da3     ONLINE       0     0     0
	  raidz1    ONLINE       0     0     0
	    ad0     ONLINE       0     0     0
	    ad1     ONLINE       0     0     0
	    ad3     ONLINE       0     0     0

errors: Permanent errors have been detected in the following files:

         /test/testfile