Re: ARM pcpu changes, was Re: [patch] i386 pmap sysmaps_pcpu[] atomic access

On Tue, Feb 26, 2013 at 09:06:51PM +1300, Andrew Turner wrote:
> Does anyone know if it is only curthread that needs to be atomic? If so
> this should work. Reading the cpuid from the system currently is a
> single instruction, however it appears the code will need to be reworked
> for use with multiple levels of affinity, for example when there
> are clusters of cores the current code will number two cores on
> different clusters with the same id.
> 
> I don't think we need to use ldrex/strex to read/write the values, my
> understanding is the rest should be safe to access without atomic
> functions.

I read about ARM architecture some time ago. What I am saying below is not
a proposal, but rather a way for me to learn more about the architecture.

From my reading of the docs, ARM has the shadow registers, in particular,
the r13 (stack pointer) is shadowed for all privileged modes. Is the shadow
used by our kernel, is it correctly restored on the context switch ?

If yes, you can easily recover the pcb address from the current sp,
without accessing any coprocessor registers, and even without any
assignment of the global register for curthread (which needs to be
done at the kernel entry). Just align the stack
start on the 2*PAGE_SIZE boundary (like it is already done for MIPS),
and do the mask operation on the sp value to get the end of pcb.
This is atomic and context-switch safe.

pcb us already per-thread, and can store the thread pointer.
More, you can store the curcpu or cpuid pointer into pcb too,
and update it on the context switch.

amd64 has an architecturally-defined special register (k)gsbase, which
effectively must be correctly initialized at each kernel entry, and
restored on the return to usermode. Since the initialization on entry
and restoration on exit is not atomic wrt the entry/exit itself, amd64
periodically gets a bugs which cause kernel running with user gsbase.
This is the fatal bug, destroying the kernel structures and allowing the
CPU privilege mode switch for normal user.

Using the shadow registers for this purpose eliminate the whole source
of the bugs.