Re: r346025: ZFS filesystems do not mount anymore

On 2019-04-09 09:30, O. Hartmann wrote:
> On Tue, 9 Apr 2019 10:42:47 +0200 (CEST)
> Trond Endrestøl <Trond.Endrestol_at_fagskolen.gjovik.no> wrote:
> 
>> On Tue, 9 Apr 2019 10:06+0200, Trond Endrestøl wrote:
>>
>>> On Mon, 8 Apr 2019 19:58+0200, O. Hartmann wrote:
>>>   
>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>> Hash: SHA256
>>>>
>>>> Hello,
>>>>
>>>> after a bunch of updates this weekend, mounting ZFS filesystems on
>>>> CURRENT r346025 doesn't work anymore at boot time when ZFS is
>>>> built-in-kernel. zfs_enable="YES" is set in /etc/rc.conf.
>>>>
>>>> After the system has booted, mounting all ZFS filesystems via "zfs mount
>>>> -a" operates as expected and all filesystems are available as usual.  
>>>
>>> I blame r346017:
>>>
>>> Trying to mount root from zfs:zroot/ROOT/20190409-r346017 []
>>> Setting hostuuid: 7b624d1c-4bc3-e2cb-4ac2-6487a4b4774c.
>>> Setting hostid: 0xf9071336.
>>> interface zfsctrl.1 already present in the KLD 'kernel'!
>>> linker_load_file: /boot/kernel/zfs.ko - unsupported file type
>>> kldload: an error occurred while loading module zfs. Please check dmesg(8)
>>> for more details. /etc/rc: WARNING: Unable to load kernel module zfs
>>> Starting file system checks:
>>> Mounting local filesystems:.
>>> interface zfsctrl.1 already present in the KLD 'kernel'!
>>> linker_load_file: /boot/kernel/zfs.ko - unsupported file type
>>> kldload: an error occurred while loading module zfs. Please check dmesg(8)
>>> for more details. /etc/rc: WARNING: Unable to load kernel module zfs
>>> interface zfsctrl.1 already present in the KLD 'kernel'!
>>> linker_load_file: /boot/kernel/zfs.ko - unsupported file type
>>> kldload: an error occurred while loading module zfs. Please check dmesg(8)
>>> for more details. /etc/rc: WARNING: Unable to load kernel module zfs
>>> ELF ldconfig path: /lib /usr/lib /usr/lib/compat
>>> 32-bit compatibility ldconfig path: /usr/lib32
>>>
>>> [...]
>>>
>>> Creating and/or trimming log files.
>>> Starting syslogd.
>>> Apr  9 09:48:32 <kern.emerg> freebsd-head-zfs syslogd: /var/log/security:
>>> No such file or directory Setting date via ntp.
>>>  9 Apr 09:48:38 ntpdate[1073]: step time server 2001:W:X:Y::Z offset
>>> -0.992370 sec No core dumps found.
>>> Clearing /tmp (X related).
>>> Updating motd:.
>>> Mounting late filesystems:mount: /usr/compat/linux: No such file or
>>> directory mount: /usr/compat/linux: No such file or directory
>>> mount: /usr/compat/linux: No such file or directory
>>> mount: /usr/compat/linux: No such file or directory
>>> .
>>> Mounting /etc/fstab filesystems failed,  startup aborted
>>> ERROR: ABORTING BOOT (sending SIGTERM to parent)!
>>> Enter full pathname of shell or RETURN for /bin/sh:
>>> root_at_freebsd-head-zfs:/ # zfs mount -av
>>> root_at_freebsd-head-zfs:/ # mount -al
>>> root_at_freebsd-head-zfs:/ # exit
>>> Setting hostuuid: 7b624d1c-4bc3-e2cb-4ac2-6487a4b4774c.
>>> Setting hostid: 0xf9071336.
>>> interface zfsctrl.1 already present in the KLD 'kernel'!
>>> linker_load_file: /boot/kernel/zfs.ko - unsupported file type
>>> kldload: an error occurred while loading module zfs. Please check dmesg(8)
>>> for more details. /etc/rc: WARNING: Unable to load kernel module zfs
>>> Fast boot: skipping disk checks.
>>> Mounting local filesystems:.
>>> interface zfsctrl.1 already present in the KLD 'kernel'!
>>> linker_load_file: /boot/kernel/zfs.ko - unsupported file type
>>> kldload: an error occurred while loading module zfs. Please check dmesg(8)
>>> for more details. /etc/rc: WARNING: Unable to load kernel module zfs
>>> interface zfsctrl.1 already present in the KLD 'kernel'!
>>> linker_load_file: /boot/kernel/zfs.ko - unsupported file type
>>> kldload: an error occurred while loading module zfs. Please check dmesg(8)
>>> for more details. /etc/rc: WARNING: Unable to load kernel module zfs
>>> ELF ldconfig
>>> path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/compat/pkg /usr/local/lib/perl5/5.28/mach/CORE
>>> 32-bit compatibility ldconfig path: /usr/lib32
>>>
>>> [...]  
>>
>> At a later point, this happens:
>>
>> Starting syslogd.
>> usage: protect [-i] command
>>        protect [-cdi] -g pgrp | -p pid
>> Setting date via ntp.
>>
>> It looks like syslogd is no longer protected against the OOM killer.
>>
>>> Once I manually mount the remaining filesystems, multiuser boot 
>>> proceeds as expected. My kernel has options ZFS, and the boot scripts 
>>> doesn't account for this scenario.  
>>
>>
> 
> Hello,
> 
> thank you for investigating! I think you're correct. I was wondering how this
> change could make it into the tree with this kind of impact.
> 
> Maybe filing a PR would be fine?
> 

I just confirmed that the change at SVN r346017 after 'mergemaster -U'
yields a (ZFS-based) system which can only be brought into normal
service through:

	a single-user boot
	mount -a
	mount -u -o rw /
	zfs mount -a

Reverting this change restores normal operation at start-up.

ZFS is also compiled into the kernel on my system,

	imb