Re: KLD zfs.ko: depends on kernel - not available or version mismatch

On Tue, Dec 08, 2020 at 07:10:26PM +0100, Alban Hertroys wrote:
> > You didn't say that you've installed the new kernel, which at least starts
> > you down the road towards a driver/kernel mismatch.  You presumably have a
> > non-ZFS boot+root.
> 
> I???m fairly sure I did, actually.
> 
> Last time I checked, "make buildworld buildkernel" was equivalent to "make buildworld && make buildkernel", while "make kernel??? is a shorthand for ???make buildkernel && make installkernel???
> 
> So, unless I???m mistaken, ???make buildworld kernel??? should be equivalent to your first two lines.
> 
> Nevertheless, I retried without these assumptions, the result was the same. I forgot to ???make delete-old??? though, I rarely remember to do that???

Ah, the dangers of command syntax being close to human syntax.  You're trying
to do the right thing, so maybe we can sanity check that.

> I had to copy over several files from /etc and /usr/local/etc and re-installed the most important packages. This was admittedly a bit messy, it is possible that I forgot to copy something over.
> (Originally my intention was to dd the contents of the spinning disk over, but apparently that disk has a few wonky sectors, dd failed after a few device timeouts)

  ... so, no guarantee that things are totally sane.

The "sane" we're looking for is how you can presumably be booting a kernel
located at /boot/kernel/kernel and not have it match the kernel modules
found under /boot/kernel.

The fact that it is happy with the old kernel modules (presumably under found
in /boot/kernel.old) may be a red herring if they're just compatible enough.


I can see what I'm expecting to boot here:

	# grep -E 'boot\/kernel|f7b0aedd1c50' /var/log/messages | tail -2
	Dec  6 08:59:04 ouroboros syslogd: kernel boot file is /boot/kernel/kernel
	Dec  6 08:59:04 ouroboros kernel: FreeBSD 13.0-CURRENT #237 r368388+f7b0aedd1c50-c273383(master): Sun Dec  6 08:27:47 PST 2020


So, I build my system with WITHOUT_REPRODUCIBLE_BUILD=YES in /etc/src.conf,
so I can easily see my build version with uname -v:

	FreeBSD 13.0-CURRENT #237 r368388+f7b0aedd1c50-c273383(master): ...

That matches my source tree:

	# git log -n1 /usr/src | grep revision
	    svn path=/head/; revision=368388

(I've always used git for my sources, but I'm sure there is a svn equivalent.)

The version I'm running is what and where I'd expect it to be:

	# strings -a < /boot/kernel/kernel | grep 'FreeBSD 13' | tail -1
	FreeBSD 13.0-CURRENT #237 r368388+f7b0aedd1c50-c273383(master): Sun Dec  6 08:27:47 PST 2020

It certainly isn't the previous kernel:

	# strings -a < /boot/kernel.old/kernel | grep 'FreeBSD 13' | tail -1
	FreeBSD 13.0-CURRENT #236 r368353+0252bfaea893-c273359(master): Fri Dec  4 16:55:41 PST 2020

Not sure what that'll look like with reproducible builds.  The hash-check below
is a decent stamp, in case the timestamps in /boot/kernel are misleading.

What I have built in my source tree is the kernel/zfs module I'd expect:

	# md5 -r /usr/obj/usr/src/amd64.amd64/sys/GENERIC/kernel /usr/obj/usr/src/amd64.amd64/sys/GENERIC/modules/usr/src/sys/modules/zfs/zfs.ko /boot/kernel/kernel /boot/kernel/zfs.ko | sort
	941ab52d075e444da6eea7fb56213e10 /boot/kernel/kernel
	941ab52d075e444da6eea7fb56213e10 /usr/obj/usr/src/amd64.amd64/sys/GENERIC/kernel
	97d4e0c8ffed1f75e924bf8768a95ff1 /boot/kernel/zfs.ko
	97d4e0c8ffed1f75e924bf8768a95ff1 /usr/obj/usr/src/amd64.amd64/sys/GENERIC/modules/usr/src/sys/modules/zfs/zfs.ko

What are you seeing after your installkernel equivalent?

Your hashes won't match mine due to non-reproducible build.

I'd make sure you don't have anything in /boot/modules or otherwise load any
extra modules until sanity is restored (just to reduce random variables).