Re: best approximation of getcpu() ?

On Fri, Dec 16, 2016 at 03:17:19AM +0100, Luigi Rizzo wrote:

> TL;DR; is there any way a userspace thread in FreeBSD can tell
> on which CPU it is (was) running ? I know the thread can migrate
> at any time but as long as the event is rare I can live with
> the occasionally wrong information.
> Linux has getcpu(2) which is exposed by glibc as sched_getcpu(3),
> but the linuxulator in FreeBSD only has a dummy (uniplemented)
> function for that syscall.
> 
> FULL DESCRIPTION
> It is common practice, when building scalable systems, to use per-cpu
> resources that can be accessed without contention by just protecting
> them with a CLI; STI; pair. Multiqueue NICs typically do this to
> build a lock-free transmit path. In [1] we show an in-kernel
> scheduler that maps a large number of clients to a (much smaller)
> number of lock-free mailboxes, one per core.
> 
> In the kernel we can do CLI and STI and access curcpu.
> In userspace a suitably privileged process can indeed open /dev/io
> to acquire the right to use CLI and STI, though I am not sure
> wether curcpu is available in some way.
> 
> Of course running userspace code with interrupts disabled is risky,
> but we can use the per-cpu trick with a small tweak, namely,
> protect each resouce with a lock. If the thread does not migrate
> imediately after getcpu(), we will access the lock (and the resource)
> almost always from the same cpu hence very efficiently.
> Occasional migration may cause contention but should not
> alter too much the good performance of this scheme.
> 
> So, any idea before I add a syscall/ioctl (or extend one)
> to export this information ?

Most hard task is discover what irq handle specific NIC tx/rx queue
(assume irq already have affinity).
After this discover (by some magic shit) no problem to start pthread
and execute some like

    CPU_ZERO(&mask);
    CPU_SET(i, &mask);
    pthread_attr_init(&attr);
    pthread_attr_setaffinity_np(&attr, sizeof(mask), &mask);
    pthread_create(&balancer[i].tid, &attr, balancer_thread, &balancer[i]);
    pthread_set_name_np(balancer[i].tid, name);