Re: Encrypted zfs?

On Thu, Sep 06, 2007 at 11:42:22AM -0500, Barry Pederson wrote:
> Pawel Jakub Dawidek wrote:
> >On Thu, Aug 30, 2007 at 03:56:35PM +0900, Nathan Butcher wrote:
> >>>AFAIK zfs is immune against device enumeration issues itself. There is a
> >>>nice video on YouTube showing Sun engineers setting up a ZFS pool on a
> >>>bunch of USB sticks. Afterwards they remove all of them, shuffle them,
> >>>and put them back in. No problem.
> >>You're correct,... only as long as the zpool is EXPORTED FIRST, and
> >>imported after the drives have been shuffled around. ZFS has no trouble
> >>piecing them back together wherever they are during an import, it seems.
> >>
> >>If you were to, say, forget to export the zpool, shutdown your system,
> >>shuffle the drives around, and THEN restart the system with the drives
> >>in the wrong places, zfs will consider the zpool unavailable. In this
> >>case, all the drives will be turn up as FAULTED due to "corrupted
> >>data"... when in reality, ZFS was set up to expect certain data to be on
> >>certain drives, and now it just can't find it thanks to the harddrive
> >>"hokey-pokey" done on it.
> >>
> >>I guess glabeling isn't really necessary, but it does prevent the above
> >> issue from ever occuring.... "An ounce of prevention" or something like
> >>that.
> >
> >You are correct, but not entirely. If you don't export the pool before
> >shuffling driver around, ZFS can still recognize them after reboot, but
> >those drives have to support GEOM::ident attribute. A disk, when asked
> >about this attribute, returns its serial number. If ZFS can find disk
> >using its name, it tries to use its ident. Not all GEOM providers
> >support idents. Currently only ATA disks and slices/partitions on top of
> >ATA disks.
> 
> I've been fooling with a zpool of 8 glabeled SATA disks, 4 of which are 
> connected to motherboard SATA ports, and 4 to SAS ports.
> 
> After shutting the machine down (without exporting the pool) and 
> shuffling disks around , it seems that the disks that were originally 
> connected via SATA but that are now on the SAS controller show up as 
> "UNAVAIL"/"cannot open", and doing a "zpool online tank label/750_03" 
> (for example) doesn't have any effect.  The disks really are present 
> though, showing up in dmesg, glabel status, and able do "dd" from them.
> 
> The disks that were originally on the SAS controller when the pool was 
> created show up fine when connected to the motherboard SATA ports.
> 
> Doing a "strings /boot/zfs/zpool.cache" I see that it's storing the 
> serial numbers as described above, of the originally SATA-connected pool 
> members, such as:
> 
>   path
>   /dev/label/750_03
>   devid
>   5QD01BS6s0
> 
> but the originally SAS-connected drives don't show that "devid" (which 
> makes sense).
> 
> Does ZFS ignore the path if it thinks the device should have a certain 
> devid?  Is there a slick way to clear out the devid/serial-number from 
> the zpool's memory?

Have you tried zpool export/import?

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd_at_FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!