ZFS fails with bsdinstaller 9.0RC3

Hi,

I just tried to install FreeBSD 9.0RC3 with a ZFS-only file system. I
succeeded by doing a manual install. The bsdinstaller failed: it would
write not write the new filesystem to /mnt as expected (I presume it has
overwritten the memstick filesystem at /). I'm relative new to FreeBSD,
so I'm trying to understand what I did wrong, or if this is quirk in the
bsdinstaller (unlikely).

This is how it failed:

- ----- start log #1 --------
When bsdinstall asks how to partition the disk, drop to the shell.

Use this shell to set up partitions for the new system. When finished,
mount the system at /mnt and place an fstab file for the new system at
/tmp/bsdinstall_etc/fstab. Then type 'exit'. You can also enter the
partition editor at any time by entering 'bsdinstall partedit'.

<previously>
# gpart add -t freebsd-zfs -l ssd0 ada0
# gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0
</previously>

# gpart show ada0
=>      34  30932925  ada0  GPT  (14G)
        34       128     1  freebsd-boot  (64k)
       162  30932797     2  freebsd-zfs  (14G)

# zpool list
ZFS NOTICE: Prefetch is disabled by default on i386 -- to enable,
            add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf.
ZFS WARNING: Recommended minimum kmem_size is 512MB; expect unstable
behavior.
             Consider tuning vm.kmem_size and vm.kmem_size_max
             in /boot/loader.conf.
ZFS filesystem version 5
ZFS storage pool version 28
no pools available

# zpool import
no pools available to import

# zpool create -m /mnt zroot /dev/gpt/ssd0
# zpool set bootfs=zroot zroot
# zfs set checksum=fletcher4 zroot

# zfs create zroot/var
# zfs create zroot/var/log
# zfs set compression=lzjb zroot/var/log
# zfs create zroot/tmp
# chmod 1777 /mnt/tmp

# zfs create -V 2G zroot/swap
# zfs set org.freebsd:swap=on zroot/swap
# zfs set checksum=off zroot/swap

(Not creating /tmp/bsdinstall_etc/fstab since ZFS takes care of the
mounting).
# touch /tmp/bsdinstall_etc/fstab

# exit

(Proceed with bsdinstall as usual.
After running it is finished, I drop back to the shell:)

# zfs list
internal error: failed to initialize ZFS library

# ls /mnt
# ls /mnt/var
ls: /mnt/var: No such file or directory

Nothing is there, and the "internal error" made me suspicious that / was
overwritten, despite that it was mounted as read-only.
I found it also odd that /mnt/var "does not exist", even though it is a
mount point.

# mount
/dev/ufs/FreeBSD_Install on / (ufs, local, noatime, read-only)
devfs on /dev (devfs, local, multilabel)
/dev/md0 on /var (ufs, local)
/dev/md1 on /tmp (ufs, local)
zroot on /mnt (zfs, local, nfsv4acls)
zroot/var on /mnt/var (zfs, local, nfsv4acls)
zroot/var/log on /mnt/var/log (zfs, local, nfsv4acls)
zroot/tmp on /mnt/tmp (zfs, local, nfsv4acls)
- ----- end log #1 --------


Most online manuals suggest to let mount handle the mounting of disk, so
instead of
# zpool create -m /mnt zroot /dev/gpt/ssd0
I tried:
# zpool create -m legacy zroot /dev/gpt/ssd0

and created /tmp/bsdinstall_etc/fstab:

# Device        Mountpoint  FStype  Options     Dump Pass#
zroot           /           zfs     rw,noatime  0   0
zroot/swap      none        swap    sw          0   0
zroot/tmp       /tmp        zfs     rw,noatime  0   0
zroot/var       /var        zfs     rw,noatime  0   0
zroot/var/log   /var/log    zfs     rw,noatime  0   0


However, this gave the same result. /mnt was empty.


I was able to create a perfectly good file system by manually finishing
the installation process:

- ----- start log #2 --------
[continuation of log #1, just before the #exit]

# cd /usr/freebsd-dist/
# tar --unlink -xpJf base.txz -C /mnt
# tar --unlink -xpJf kernel.txz -C /mnt
# tar --unlink -xpJf ports.txz -C /mnt

# echo 'zfs_enable="YES"' >> /mnt/etc/rc.conf
# echo 'hostname="myhost.example.org"' >> /mnt/etc/rc.conf
# echo 'ifconfig_em1="DHCP"' >> /mnt/etc/rc.conf

# echo 'zfs_load="YES"' >> /mnt/boot/loader.conf
# echo 'vfs.root.mountfrom="zfs:zroot"' >> /mnt/boot/loader.conf
# echo 'console="comconsole"' >> /mnt/boot/loader.conf
# echo 'comconsole_speed="19200"' >> /mnt/boot/loader.conf

# echo '# Mounting of disk is handled by ZFS' >> /mnt/etc/fstab

# vi /etc/ttys
(enable ttyu0 and disable ttyv0 to ttyv7, since I have a Soekris, and
must resort to headless -serial console- installation, without monitor)

# chroot /mnt
# passwd
# tzsetup
# exit

# zfs umount -a
# zpool export zroot

# zpool import -o cachefile=/tmp/zpool.cache zroot
# cp /boot/zfs/zpool.cache /mnt/boot/zfs/zpool.cache

# zfs umount -a
# zfs set mountpoint=/ zroot
# /sbin/reboot now

>> success!
- ----- end log #2 --------

Any clue why the manual installation succeeded, but the bsdinstall
failed? I would presume bsdinstall would do just the same thing, only
with a (much) nicer GUI.

Regards,
Freek