Re: [RFC] Start SMP subsystem earlier

From: Konstantin Belousov <kostikbel_at_gmail.com>
Date: Tue, 6 Jan 2015 17:12:11 +0200
On Tue, Jan 06, 2015 at 09:37:30AM -0500, John Baldwin wrote:
> On 1/5/15 8:18 AM, Hans Petter Selasky wrote:
> > Hi,
> > 
> > There is a limitiation on the number of interrupt vectors available when
> > only a single processor is running. To have more interrupts available we
> > need to start SMP earlier when building a monotolith kernel and not
> > loading drivers as modules. The driver in question is a network driver
> > and because it cannot be started after SI_SUB_ROOT_CONF due to PXE
> > support I see no other option than to move SI_SUB_SMP earlier.
> > 
> > Suggested patch:
> > 
> >> Index: sys/kernel.h
> >> ===================================================================
> >> --- sys/kernel.h    (revision 276691)
> >> +++ sys/kernel.h    (working copy)
> >> _at__at_ -152,6 +152,7 _at__at_
> >>      SI_SUB_KPROF        = 0x9000000,    /* kernel profiling*/
> >>      SI_SUB_KICK_SCHEDULER    = 0xa000000,    /* start the timeout
> >> events*/
> >>      SI_SUB_INT_CONFIG_HOOKS    = 0xa800000,    /* Interrupts enabled
> >> config */
> >> +    SI_SUB_SMP        = 0xa850000,    /* start the APs*/
> >>      SI_SUB_ROOT_CONF    = 0xb000000,    /* Find root devices */
> >>      SI_SUB_DUMP_CONF    = 0xb200000,    /* Find dump devices */
> >>      SI_SUB_RAID        = 0xb380000,    /* Configure GEOM classes */
> >> _at__at_ -165,7 +166,6 _at__at_
> >>      SI_SUB_KTHREAD_BUF    = 0xea00000,    /* buffer daemon*/
> >>      SI_SUB_KTHREAD_UPDATE    = 0xec00000,    /* update daemon*/
> >>      SI_SUB_KTHREAD_IDLE    = 0xee00000,    /* idle procs*/
> >> -    SI_SUB_SMP        = 0xf000000,    /* start the APs*/
> >>      SI_SUB_RACCTD        = 0xf100000,    /* start racctd*/
> >>      SI_SUB_LAST        = 0xfffffff    /* final initialization */
> >>  };
> > 
> > This fixes a problem for Mellanox drivers in the OFED layer. Possibly we
> > need to move the SMP even earlier to not miss the generic FreeBSD PCI
> > device enumeration or maybe this is not possible. Does anyone know how
> > early we can start SMP?
> 
> We need a lot more work before this is ready.  This is one of the goals
> of the multipass new-bus stuff.  In particular, we have to enumerate
> enough devices to bring event timer hardware up so that timer interrupts
> work so that tsleep() will actually sleep.  In addition, we also need
> idle threads created and working before APs are started as otherwise
> they will have no thread to run initially.  This is certainly a desired
> feature, but it is not as simple as moving the sysinit up I'm afraid.
> 
I believe that idle threads are still created before the APs start
with the patch posted, this was the thing I checked first. It is
SUB_SCHED_IDLE, which is done long before even drivers are configured.
Received on Tue Jan 06 2015 - 14:12:17 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:54 UTC