Re: lapic_at_2k interrukts eating CPU cycles

Am Mittwoch, 22. Juni 2005 16:50 schrieb Robert Watson:
> On Wed, 22 Jun 2005, Emanuel Strobl wrote:
> >> Because you have 'options APIC' in your kernel, and your running a
> >> post 5.2 version of FreeBSD that has a massively improved interrupt
> >> routing system in it.
> >
> > I've been running 5-current on that machine with "options APIC" for a
> > long time, back to when I had to include NO-MIXEDMODE to get it
> > working until I think Robeert Watson gave me some patches. But I never
> > had lapic so I was wondering...
>
> Might have been John Baldwin.

Hm, me and names is a weak point... ;)

> In 5.x, timer interrupts are generated by a general purpose programmable
> timer and delivered as interrupts to one of your CPUs on an MP system,
> or your only CPU on a UP system.  Depending on your hardware, where the
> timer went varied a bit -- typically, round robin, or maybe your first
> CPU or third CPU.  The timer tick would then be broadcasted to the other
> CPUs using an inter-processor interrupt (IPI), so that all CPUs would
> see it for the purposes of timer-based preemption, profiling,
> statistics, and so on.
>
> In 6.x, we use the timer in the CPU-local apic (LAPIC) to interrupt each
> CPU, rather than delivering all the interrupts to a particular CPU and
> forwarding them.  This lowers the IPI traffic overhead, which can be
> substantial due to synchronizing and preventing IPI deadlock.  In
> particular, spin locks are used around IPI interactions to prevent two
> or more CPUs from deadlocking by sending each other IPIs simultaneously,
> and therefore being unable to acknowledge each other (there's a whole
> literature on why and how to do this, if you google or check a decent MP
> OS text).
>
> So in 5.x, you saw the primary timer interrupt traffic explicitly, and
> most interrupt monitoring tools didn't monitor IPI load.  In 6.x, you
> see the timer interrupt traffic on each CPU as lapic load for the CPU. 
> In 6.x, we currently don't monitor IPI interrupt load, but if we did in
> 5.x and 6.x, you'd see that the IPI traffic level is much lower, making
> up for the increased LAPIC interrupt traffic.
>
> I have a feature request in to John to add statistics gathering on IPIs,
> since he's currently reworking the interrupt paths.
>
> Robert N M Watson

Thanks so much, very good explanation, I almost understood it completely 
(I'm lacking a lot of basics). If only "The Design and Implementation of 
the FreeBSD Operating System" was written like this...

-Harry