Re: rc.d/zpool runs before ada(4) attaches

 
Van: Ian Lepore <ian_at_freebsd.org>
Datum: dinsdag, 1 december 2020 16:34
Aan: Ronald Klop <ronald-lists_at_klop.ws>, FreeBSD Current <freebsd-current_at_freebsd.org>
Onderwerp: Re: rc.d/zpool runs before ada(4) attaches
> 
> On Tue, 2020-12-01 at 16:22 +0100, Ronald Klop wrote:
> >  
> > Van: Harry Schmalzbauer <freebsd_at_omnilan.de>
> > Datum: dinsdag, 1 december 2020 12:51
> > Aan: Ronald Klop <ronald-lists_at_klop.ws>, FreeBSD Current <
> > freebsd-current_at_freebsd.org>
> > Onderwerp: Re: rc.d/zpool runs before ada(4) attaches
> > >
> > > Am 01.12.2020 um 10:33 schrieb Ronald Klop:
> > > :
> > > :
> > > :
> > > > > One machine fails importing zpool because the correponding
> > > > > vdevs >> (ada0-ada2)
> > > > > are not available at the time rc.d/zpool runs.
> > > > >
> > > > >
> > > > > Adhoc  I'm not aware of any rc(8) vs. driver awareness.
> > > > > Is there any?
> > > > >
> > > > > Suggestions how to fix else than 'sleep 1'?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > -harry
> > > > >
> > > > > P.S.: ahci(4) is compiled into kernel, machine is a HPE U48
> > > > > (Gen 10 >> plus MicroServer), zfsloader loads root_MFS kernel
> > > > > module
> > > > >
> > > >
> > > >
> > > > There have been some changes to etc/rc.d/zpool in September.
> > > > Do you have the latest version? Compare with:
> > > >
> https://github.com/freebsd/freebsd/blob/master/libexec/rc/rc.d/zpool
> > > > or
> > > >
> https://svnweb.freebsd.org/base/head/libexec/rc/rc.d/zpool?revision=365354&view=markup
> > > >  >
> > > >
> > > > Otherwise it would be helpful for readers if you could post some
> > > > logs > which indicate what is happening.
> > > > /var/run/dmesg.boot or the output of "dmesg"
> > > > Part of /var/log/messages
> > > > Part of /var/log/console.log if it exists.
> > > >
> > >
> > > Thanks, I'm on -current from view days ago.
> > > The problem is that cam(4) is still probing devices, when
> > > rc.d/zpool runs, since mount_root_from succeeded, because it is a
> > > RAM disk, so succeeds independent of any real drive/controller
> > > probing.
> > > I can imagine of other seldom edgecases hitting the issue too.
> > >
> > > So my proposed patch, working for me, looks like this:
> > > Index: libexec/rc/rc.d/zpool
> > > ===================================================================
> > > --- libexec/rc/rc.d/zpool       (revision 368202)
> > > +++ libexec/rc/rc.d/zpool       (working copy)
> > > _at__at_ -18,8 +18,16 _at__at_
> > >
> > >   zpool_start()
> > >   {
> > > -       local cachefile
> > > +        local cachefile n=0 camlist=`camcontrol devlist -v`
> > >
> > > +       # Wait for cam(4) devices attaching, 4 times at max by
> > > increasing
> > > +       # 1s each (10s max in total)
> > > +        while [ X"${camlist#*target*lun*probe}" != X"${camlist}"
> > > ]; do
> > > +               [ $n -lt 4 ] || break
> > > +               sleep $((n+=1))
> > > +               camlist=`camcontrol devlist -v`
> > > +       done
> > > +
> > >          for cachefile in /etc/zfs/zpool.cache
> > > /boot/zfs/zpool.cache; do
> > >                  if [ -r $cachefile ]; then
> > >                          zpool import -c $cachefile -a -N && break
> > >
> > > best,
> > > -harry
> > >
> > >
> > >
> >
> > You can define these in /boot/loader.conf:
> > #kern.cam.boot_delay="10000" # Delay (in ms) of root mount for CAM
> > bus
> > #kern.cam.scsi_delay="2000" # Delay (in ms) before probing SCSI
> >
> > Maybe that helps.
> >
> > Ronald.
> >
> 
> Those settings control waiting before mounting root.  Harry's problem
> is that root is mounted quickly, before other drives are ready for zfs.
>  
> The zpool script waits for 'disks'.  It would be nice if the cam
> subsystem had something like a sysctl it set to indicate when initial
> probing for disks was done, then there could be an rc.d/camprobe script
> with 'PROVIDE: disks' which waits for the probing to complete.
> 
> -- Ian
> 
>  
> 
> 
> 

Or a devd event "ada-arrived" which calls rc.d/zpool.
It sounds a bit like we need systemd. :-)

Ronald.