Re: ARM pcpu changes, was Re: [patch] i386 pmap sysmaps_pcpu[] atomic access

On Tue, Feb 26, 2013 at 1:39 PM, Konstantin Belousov
<kostikbel_at_gmail.com> wrote:
> On Tue, Feb 26, 2013 at 09:06:51PM +1300, Andrew Turner wrote:
>> Does anyone know if it is only curthread that needs to be atomic? If so
>> this should work. Reading the cpuid from the system currently is a
>> single instruction, however it appears the code will need to be reworked
>> for use with multiple levels of affinity, for example when there
>> are clusters of cores the current code will number two cores on
>> different clusters with the same id.
>>
>> I don't think we need to use ldrex/strex to read/write the values, my
>> understanding is the rest should be safe to access without atomic
>> functions.
>
> I read about ARM architecture some time ago. What I am saying below is not
> a proposal, but rather a way for me to learn more about the architecture.
>
> From my reading of the docs, ARM has the shadow registers, in particular,
> the r13 (stack pointer) is shadowed for all privileged modes. Is the shadow
> used by our kernel, is it correctly restored on the context switch ?
>
> If yes, you can easily recover the pcb address from the current sp,
> without accessing any coprocessor registers, and even without any
> assignment of the global register for curthread (which needs to be
> done at the kernel entry). Just align the stack
> start on the 2*PAGE_SIZE boundary (like it is already done for MIPS),
> and do the mask operation on the sp value to get the end of pcb.
> This is atomic and context-switch safe.
>
> pcb us already per-thread, and can store the thread pointer.
> More, you can store the curcpu or cpuid pointer into pcb too,
> and update it on the context switch.
>
> amd64 has an architecturally-defined special register (k)gsbase, which
> effectively must be correctly initialized at each kernel entry, and
> restored on the return to usermode. Since the initialization on entry
> and restoration on exit is not atomic wrt the entry/exit itself, amd64
> periodically gets a bugs which cause kernel running with user gsbase.
> This is the fatal bug, destroying the kernel structures and allowing the
> CPU privilege mode switch for normal user.
>
> Using the shadow registers for this purpose eliminate the whole source
> of the bugs.

Well, IMHO, the both kernel and user thread stacks must correctly be
set for current threads, otherwise it doesn't work. So, the idea to
save things on a thread stack and update them on a context switch will
work in general. However, the stack must be aligned to its size. And
this is the price. I have no idea how this kernel stack alignment can
impact kernel virtual space fragmentation.
OTOH, as you say, this kernel stack storage can be used for many things.

Svata