Re: best approximation of getcpu() ?

On 16 December 2016 at 11:45, Luigi Rizzo <rizzo_at_iet.unipi.it> wrote:
> On Fri, Dec 16, 2016 at 09:29:15AM +0000, David Chisnall wrote:
>> On 16 Dec 2016, at 03:10, Alan Somers <asomers_at_FreeBSD.org> wrote:
>> >
>> > What about pthread_setaffinity(3) and friends?  You can use it to pin
>> > a thread to a single CPU, and know that it will never migrate.
>>
>> This is not a useable solution for anything that needs to live in a library and also doesn???t solve the problem.
>>
>> The Linux get_cpu call() is used for caches that are somewhere between global and thread-local.  Accessing them still requires a lock, but it???s very likely to be uncontended (contention only happens when you???re context switched at exactly the wrong time, or if a thread is migrated between cores in between the get_cpu() call and usage) and so you can use the userspace fast path for the lock and not suffer from cache contention effects.
>>
>> One x86, you can use cpuid from userspace and get the current core ID.  I have some code that does this and re-checks every few hundred accesses, storing the current CPU ID in a thread-local variable.  Using the per-CPU caches is a lot faster than using the global cache (and reduces contention on the global cache).  It would be great if we could have a syscall to do this on FreeBSD (it would be even better if we could have specify a TLS variable that the kernel automatically updates for the userspace thread when the scheduler migrates the thread between cores).
>
> indeed the following line seems to do the job for x86
>         asm volatile("cpuid" : "=d"(curcpu), "=a"(tmp), "=b"(tmp), "=c"(tmp) : "a"(0xb) );
> (there must be a better way to tell the compiler that eax, ebx, ecx, edx are
> all clobbered).
>
> 0xb is the CPUID function that returns the current APIC id for the
> core (not necessarily matching the OS core-id)
>
> The only problem is that this instruction is serialising and slow,
> seems to take some 70-100ns on several of my machines so you
> cannot afford to call it at all times but need the value cached
> somewhere. Exposing it as thread local storage, or a VDSO syscall,
> would be nicer because the kernel knows when it is actually changing
> value.

The problem is your CPU ID can change in the middle of packet handling.

So if you want it to be accurate, you need to bind your worker thread to a CPU.



-adrian