Re: [HEADSUP] zfs root pool mounting

On Thu, Dec 6, 2012 at 4:33 PM, Garrett Cooper <yanegomi_at_gmail.com> wrote:
> On Thu, Dec 6, 2012 at 3:08 PM, Garrett Cooper <yanegomi_at_gmail.com> wrote:
>
> ...
>
>> Please document the process to make this work in UPDATING (or at least
>> the fact that this behavior was changed).
>>
>> I'm debugging moving from 9.1-RC2 to CURRENT [as of Tuesday] as it
>> hasn't been as smooth as some of the other upgrades I've done; my
>> zpool -- root -- is setup with a non-legacy mountpoint, I noticed that
>> the cachefile attribute is now "None", etc. I have limited capability
>> with my installed system to debug this because unfortunately there
>> aren't a ton of CURRENT based livecds around to run from (I might look
>> into one of gjb's livecds later on if I get super stuck, but I'm
>> trying to avoid having to do that). gptzfsboot sees the pool with
>> lsdev, but it gets stuck at the mountroot prompt trying to find the
>> filesystem.
>>
>> I'll wipe my /boot/kernel directory and try building/installing the
>> kernel again, but right now I'm kind of dead in the water on the
>> system I'm upgrading :/.
>
>     I thought r236884 requiring a zpool upgrade was the culprit, but
> it wasn't. Still stuck at a mountroot prompt (but now I have gjb's
> liveCD so I can do something about it).
>     Something looks off with zdb -l on CURRENT and STABLE/9. Example
> on my 9-stable box:
>
> # uname -a
> FreeBSD forza.west.isilon.com 9.1-PRERELEASE FreeBSD 9.1-PRERELEASE #0
> r+2fd0a57: Mon Dec  3 12:02:18 PST 2012
> gcooper_at_forza.west.isilon.com:/usr/obj/usr/src/sys/FORZA  amd64
> # zdb -l sac2
> cannot open 'sac2': No such file or directory
> # zpool list
> NAME   SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
> sac     95G  69.7G  25.3G    73%  1.00x  ONLINE  -
> sac2   232G   117G   115G    50%  1.00x  ONLINE  -
>
>     I'm running into the same behavior before and after I upgraded sac/sac2.
>     My git branch is a lightly modified version of FreeBSD, but
> doesn't contain any ZFS specific changes (I can point you to it if you
> like to look at it).
>     Would appreciate some pointers on what to do next.

(Removing bogus list)

If I try and let it import the pool at boot it claims the pool is in a
FAULTED state when I point mountroot to /dev/cd0 (one of gjb's
snapshot CDs -- thanks!), run service hostid onestart, etc. If I
export and try to reimport the pool it claims it's not available (!).
However, if I boot, run service hostid onestart, _then_ import the
pool, then the pool is imported properly.

While I was mucking around with the pool trying to get the system to
boot I set the cachefile attribute to /boot/zfs/zpool.cache before
upgrading. In order to diagnose whether or not that was at fault, I
set that back to none and I'm still running into the same issue.

I'm going to try backing out your commit and rebuild my kernel in
order to determine whether or not that's at fault.

One other thing: both my machines have more than one ZFS-only zpool,
and it might be probing the pools in the wrong order; one of the pools
has bootfs set, the other doesn't, and the behavior is sort of
resembling it not being set properly.

Thanks,
-Garrett