Re: Encrypted zfs?

Pawel Jakub Dawidek wrote:
> On Thu, Aug 30, 2007 at 03:56:35PM +0900, Nathan Butcher wrote:
>>> AFAIK zfs is immune against device enumeration issues itself. There is a
>>> nice video on YouTube showing Sun engineers setting up a ZFS pool on a
>>> bunch of USB sticks. Afterwards they remove all of them, shuffle them,
>>> and put them back in. No problem.
>> You're correct,... only as long as the zpool is EXPORTED FIRST, and
>> imported after the drives have been shuffled around. ZFS has no trouble
>> piecing them back together wherever they are during an import, it seems.
>>
>> If you were to, say, forget to export the zpool, shutdown your system,
>> shuffle the drives around, and THEN restart the system with the drives
>> in the wrong places, zfs will consider the zpool unavailable. In this
>> case, all the drives will be turn up as FAULTED due to "corrupted
>> data"... when in reality, ZFS was set up to expect certain data to be on
>> certain drives, and now it just can't find it thanks to the harddrive
>> "hokey-pokey" done on it.
>>
>> I guess glabeling isn't really necessary, but it does prevent the above
>>  issue from ever occuring.... "An ounce of prevention" or something like
>> that.
> 
> You are correct, but not entirely. If you don't export the pool before
> shuffling driver around, ZFS can still recognize them after reboot, but
> those drives have to support GEOM::ident attribute. A disk, when asked
> about this attribute, returns its serial number. If ZFS can find disk
> using its name, it tries to use its ident. Not all GEOM providers
> support idents. Currently only ATA disks and slices/partitions on top of
> ATA disks.

I've been fooling with a zpool of 8 glabeled SATA disks, 4 of which are 
connected to motherboard SATA ports, and 4 to SAS ports.

After shutting the machine down (without exporting the pool) and 
shuffling disks around , it seems that the disks that were originally 
connected via SATA but that are now on the SAS controller show up as 
"UNAVAIL"/"cannot open", and doing a "zpool online tank label/750_03" 
(for example) doesn't have any effect.  The disks really are present 
though, showing up in dmesg, glabel status, and able do "dd" from them.

The disks that were originally on the SAS controller when the pool was 
created show up fine when connected to the motherboard SATA ports.

Doing a "strings /boot/zfs/zpool.cache" I see that it's storing the 
serial numbers as described above, of the originally SATA-connected pool 
members, such as:

   path
   /dev/label/750_03
   devid
   5QD01BS6s0

but the originally SAS-connected drives don't show that "devid" (which 
makes sense).

Does ZFS ignore the path if it thinks the device should have a certain 
devid?  Is there a slick way to clear out the devid/serial-number from 
the zpool's memory?

	Barry