Re: KLD zfs.ko: depends on kernel - not available or version mismatch

On Tue, Dec 08, 2020 at 08:56:25AM +0100, Alban Hertroys wrote:
> This seems to have gotten lost in the moderate queue, but after a week I am no closer to a solution, so here???s a resend:
> 
> I???ve been trying to get a fresh world running (for the eventual purpose of running amdgpu against my recent graphics adapter), but I run into trouble with core loadable kernel modules, such as zfs.ko from the subject. It also happens with other modules that I tried randomly, for example, geom_mirror.ko.
> 
> I updated to the latest current using svn up in /usr/src, then:
> 	make clean
> 	make buildworld kernel -j12
> 	shutdown -r now
> 
> boot to single user mode
> 
> 	kldload zfs

I'm not sure you've provided enough information for a one-shot armchair
diagnosis, but some things seem factually wrong.  For example, my normal
rebuild procedure is:

	cd /usr/src && make buildworld && make buildkernel
	make installkernel
	shutdown -r now

	cd /usr/src && mergemaster -pi
	make installworld
	mergemaster -Fi
	make -DBATCH_DELETE_OLD_FILES delete-old
	shutdown -r now

	cd /usr/src && make -DBATCH_DELETE_OLD_FILES delete-old-libs

(I'm on a desktop system here.  You haven't described your setup.)

You didn't say that you've installed the new kernel, which at least starts
you down the road towards a driver/kernel mismatch.  You presumably have a
non-ZFS boot+root.  Did you mess around with the ZFS from ports (ZoL -> ZoF)
at some point so you're not using the kernel's ZFS drivers?  What ZFS
entries do you have in /etc/loader.conf, /etc/rc.conf, and some of the
varients that may get dragged in?  (see rc.conf(5) for possibilities)

At the bottom of your email, you say / is UFS and /usr is ZFS, but I guess we
have the extra fun of wondering what is under /usr on your /?  If you have a
pre-ZFS /usr that is populated by something now presumably very old (because
all the new, current stuff went onto ZFS /usr, now unavailable).

> Which results in dmesg messages:
> 
> KLD zfs.ko: depends on kernel - not available or version mismatch
> linker_load_file: /boot/kernel/zfs.ko - unsupported file type
> KLD zfs.ko: depends on kernel - not available or version mismatch
> linker_load_file: /boot/kernel/zfs.ko - unsupported file type
> KLD zfs.ko: depends on kernel - not available or version mismatch
> linker_load_file: /boot/kernel/zfs.ko - unsupported file type
> KLD zfs.ko: depends on kernel - not available or version mismatch
> linker_load_file: /boot/kernel/zfs.ko - unsupported file type

Be sure to check out /var/log/messages for extra issues.  For example, with
the bug I mentioned below, I couldn't load my nvidia driver and that manifested
as one driver having issues because it depended on another, which had the root
of the problem.

> I can load the zfs kernel module from kernel.old just fine:
> 
> ZFS filesystem version: 5
> ZFS storage pool version: features support (5000)

I kicked my more bleeding-edge system over from 12.2-rel (r366954) up into
13.0-current (r367044, 1300123) on 2020/10/26.  OpenZFS kicked in 2020/8/24?
I think the CFT was ~2018/8/21, not sure when we had the OpenZFS ports.
Current bumps the ABI version pretty frequently so I'd think you'd have
tripped across versioning issues a long time ago if you had some drivers not
being rebuilt.

> This happens with any kernel module I???ve tried, such as geom_mirror and amdgpu (from ports/graphics/drm-current-kmod - the latter causes a kernel panic with kernel.old BTW).
> 
> I???ve gone back as far as Oct 7 (before changes to kern/elf_load_obj.c off the top of my head), looked at mailing list archives and forums etc, all to no avail.
> 
> I have / on UFS+J and /usr on ZFS and nothing in /etc/src.conf. I had /etc/malloc.conf with the recommended symlink from UPDATING, but the same happens with that moved out of the way. Nothing seems to help.
> 
> Do I need to go back further to get into a usable state or is there something else I should be doing?

With very few exceptions (bug 250897, 2020/11/6), I've found 13-current
bootable since 10/26 (up through my current system, 13.0 r368388 (2020/12/6).
You obviously need to make sure that an extra drivers you add in are compiled
against the kernel, but ZFS is typically one of those.