[openzfs] r365058 arm64 uefi boot fails with "unknown filesystem" after launching kernel

hi Matt, Ryan

Something goes awry after loading kernel & zfs drivers, at point of mounting the root filesystem from boot env - unknown filesystem.

Running ? to show available fs doesn't show zfs here, but as we loaded kernel already from zfs this is confusing.

Any ideas?

mountroot> ?
List of GEOM managed disk devices:
  da1 ufs/FreeBSD_Install ufsid/5f3524aa78541663
  da0s2
  da0p1
  da0
  gpt/zfs0
  gpt/swap0
  msdosfs/EFISYS
 gpt/efiboot0
 ada0p3
 ada0p2 ada0p1 ada0

Additionally at this point, it doesn't appear to be possible to boot/mount from the previous boot environment, as zfs.ko etc is already loaded, I'm not sure if I can set this prior with `loaddev` or similar.

full dmesg/logs here - https://dpaste.org/iwRd , trimmed below. 

svn path=/head/; revision=365058 , commit 52a2c0f

Consoles: EFI console  
    Reading loader env vars from /efi/freebsd/loader.env
Setting currdev to disk1p1:
FreeBSD/arm64 EFI loader, Revision 1.1
​
   Command line arguments: loader.efi
   Image base: 0x9fe6d30000
   EFI version: 2.70
   EFI Firmware: American Megatrends (rev 5.13)
   Console: efi (0x20000000)
   Load Path: \EFI\FreeBSD\loader.efi
   Load Device: PciRoot(0xFF)/Pci(0x1,0x0)/Sata(0x0,0xFFFF,0x0)/HD(1,GPT,BE9634BF-D684-11EA-B300-001B21E07D7B,0x28,0x64000)
   BootCurrent: 0001
   BootOrder: 0001[*] 0004 001b 001f 001a 000e 0000 0003 0005 0006 0007 001c 001d 001e
   BootInfo Path: HD(1,GPT,BE9634BF-D684-11EA-B300-001B21E07D7B,0x28,0x64000)/\EFI\refind\refind_aa64.efi
Ignoring Boot0001: Only one DP found
Trying ESP: PciRoot(0xFF)/Pci(0x1,0x0)/Sata(0x0,0xFFFF,0x0)/HD(1,GPT,BE9634BF-D684-11EA-B300-001B21E07D7B,0x28,0x64000)
Setting currdev to disk1p1:
Trying: PciRoot(0xFF)/Pci(0x1,0x0)/Sata(0x0,0xFFFF,0x0)/HD(2,GPT,BE9A1581-D684-11EA-B300-001B21E07D7B,0x64800,0x2800000)
Setting currdev to disk1p2:
Trying: PciRoot(0xFF)/Pci(0x1,0x0)/Sata(0x0,0xFFFF,0x0)/HD(3,GPT,BEA23FFC-D684-11EA-B300-001B21E07D7B,0x2864800,0x355DE800)
Setting currdev to zfs:zroot/ROOT/13.0-CURRENT-20200901.193810:
Loading /boot/defaults/loader.conf
Loading /boot/defaults/loader.conf
Loading /boot/device.hints
Loading /boot/loader.conf
Loading /boot/loader.conf.local
Loading kernel...
/boot/kernel/kernel text=0x2a8 text=0x799990 text=0x1e7a6c data=0x196b80 data=0x0+0x3a378e syms=[0x8+0x165978+0x8+0x158936]
Loading configured modules...
/boot/kernel/tmpfs.ko text=0x4421 text=0x8fec data=0x1110+0x18 syms=[0x8+0x1f68+0x8+0x13df]
/boot/kernel/xz.ko text=0x850 text=0x22d0 data=0x268+0x400 syms=[0x8+0x870+0x8+0x412]
/boot/kernel/linuxkpi.ko text=0x709d text=0x13070 data=0x1970+0x750 syms=[0x8+0x5a48+0x8+0x3dfb]
/boot/entropy size=0x1000
/etc/hostid size=0x25
/boot/kernel/mlx5en.ko text=0x19b35 text=0x11a48 data=0x26d8+0x8 syms=[0x8+0x32e8+0x8+0x21ae]
/boot/kernel/mlx5.ko text=0xc369 text=0x23e24 data=0x1b50+0x94 syms=[0x8+0x5100+0x8+0x3c65]
/boot/kernel/fusefs.ko text=0x8236 text=0xcb48 data=0x3420+0x68 syms=[0x8+0x4350+0x8+0x4894]
/boot/kernel/opensolaris.ko text=0x12c3 text=0x94c data=0x468+0x6830 syms=[0x8+0x1050+0x8+0x8bb]
/boot/kernel/mlxfw.ko text=0xed2 text=0x167c data=0x258 syms=[0x8+0x750+0x8+0x409]
/boot/kernel/zfs.ko text=0xa3e88 text=0x1c8828 data=0x25908+0x158990 syms=[0x8+0x2fa48+0x8+0x29ff3]
​
Hit [Enter] to boot immediately, or any other key for command prompt.
Booting [/boot/kernel/kernel]...               
No valid device tree blob found!
WARNING! Trying to fire up the kernel, but no device tree blob found!
EFI framebuffer information:
addr, size     0x430000000, 0x1d4c00
dimensions     800 x 600
stride         800
masks          0x00ff0000, 0x0000ff00, 0x000000ff, 0xff000000
---<<BOOT>>---
KDB: debugger backends: ddb
KDB: current backend: ddb
Copyright (c) 1992-2020 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 13.0-CURRENT r365058+d8f4c1a833bf-c271116(master) GENERIC arm64
FreeBSD clang version 11.0.0 (git_at_github.com:llvm/llvm-project.git llvmorg-11.0.0-rc1-47-gff47911ddfc)
WARNING: WITNESS option enabled, expect reduced performance.
VT(efifb): resolution 800x600
module firmware already present!
KLD file zfs.ko is missing dependencies
module_register: cannot register tmpfs from kernel; already loaded from tmpfs.ko
Module tmpfs failed to register: 17
real memory  = 137167400960 (130813 MB)
avail memory = 133608034304 (127418 MB)
FreeBSD/SMP: Multiprocessor System Detected: 32 CPUs
random: unblocking device.
...
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <Micron 5200 MTFDDAK480TDC D1MH027> ACS-3 ATA SATA 3.x device
ada0: Serial Number 191020FD543B
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 457862MB (937703088 512 byte sectors)
...
sysctl_unregister_oid: failed(22) to unregister sysctl(tmpfs)
Release APs...done
Trying to mount root from zfs:zroot/ROOT/13.0-CURRENT-20200901.193810 []...
Mounting from zfs:zroot/ROOT/13.0-CURRENT-20200901.193810 failed with error 2: unknown file system.
...
da0 at umass-sim0 bus 0 scbus4 target 0 lun 0
da0: <Generic Flash Disk 8.07> Removable Direct Access SCSI-2 device
da0: Serial Number 512436A8
da0: 40.000MB/s transfers
da0: 2006MB (4108288 512 byte sectors)
da0: quirks=0x2<NO_6_BYTE>
..
Loader variables:
  vfs.root.mountfrom=zfs:zroot/ROOT/13.0-CURRENT-20200901.193810
​
Manual root filesystem specification: 
—
  Dave Cottlehuber
  +43 67 67 22 44 78
  Managing Director
  Skunkwerks, GmbH
  http://skunkwerks.at/
  ATU70126204
  Firmenbuch 410811i