Re: [patch] i386 pmap sysmaps_pcpu[] atomic access

On Thu, Feb 21, 2013 at 4:43 PM, John Baldwin <jhb_at_freebsd.org> wrote:
>> > curthread is a bit magic. :)  If you perform a context switch during an
>> > interrupt (which will change 'curthread') you also change your register state.
>> > When you resume, the register state is also restored.  This means that while
>> > something like 'PCPU_GET(cpuid)' might be stale after you read it, 'curthread'
>> > never is.  However, it is true that actually reading curthread has to be
>> > atomic.  If your read of curthread looks like:
>> >
>> >         mov <pcpu reg>, r0
>> >         add <offset of pc_curthread>, r0
>> >         ld r0, r1
>> >
>> > Then that will indeed break.  Alpha used a fixed register for 'pcpu_reg'
>> > (as does ia64 IIRC).  OTOH, you might also be able to depend on the fact that
>> > pc_curthread is the first thing in 'struct pcpu' (and always will be, you could
>> > add a CTASSERT to future-proof).  In that case, you can remove the 'add'
>> > instruction and instead just do:
>> >
>> >         ld <pcpu reg>, r1
>> >
>> > which is fine.
>>
>> Just for the record. There are three extra (coprocessor) registers per
>> a core in arm11mpcore (armv6k). Unfortunately only one is Privileged.
>> The other ones are respectively User Read only and User Read Write.
>> For now, we are using the Privileged one to store pcpu pointer
>> (curthread is correctly set during each context switch). Thus, using a
>> coprocessor register means that reading of curthread is not a single
>> instruction implementation now. After we figured out the curthread
>> issue in SMP case, using a fixed (processor) register for pcpu is an
>> option. Meanwhile, we disable interrupts before reading of curthread
>> and enable them after. The same is done for other PCPU stuff. For now
>> we have not stable system enough for profiling, however, when it will
>> be, it would be interesting to learn how various implementations of
>> curthread reading impact system performance.
>
> curthread is read _a lot_, so I would expect this to hurt.  What we did on
> Alpha was to use a fixed register for pcpu access, but we used the equivalent
> of a coprocessor register to also store the value so we could set that fixed
> register on entry to the kernel (userland was free to use the register for
> its own purposes).

Interesting solution. Thanks.

Svata