Re: Programmatically cache line

On Fri, Jan 5, 2018 at 2:29 AM, Konstantin Belousov <kostikbel_at_gmail.com>
wrote:

> On Thu, Jan 04, 2018 at 10:03:32AM +0000, David Chisnall wrote:
> > On 3 Jan 2018, at 22:12, Nathan Whitehorn <nwhitehorn_at_freebsd.org>
> wrote:
> > >
> > > On 01/03/18 13:37, Ed Schouten wrote:
> > >> 2018-01-01 11:36 GMT+01:00 Konstantin Belousov <kostikbel_at_gmail.com>:
> > >>>>>> On x86, the CPUID instruction leaf 0x1 returns the information in
> > >>>>>> %ebx register.
> > >>>>> Hm, weird. Why don't we extend sysctl to include this info?
> > >>> For the same reason we do not provide a sysctl to add two integers.
> > >> I strongly agree with Kostik on this one. Why add stuff to the kernel,
> > >> if userspace is already capable of extracting this? Adding that stuff
> > >> to sysctl has the downside that it will effectively introduce yet
> > >> another FreeBSDism, whereas something generic already exists.
> > >>
> > >
> > > Well, kind of. The userspace version is platform-dependent and not
> always available: for example, on PPC, you can't do this from userland and
> we provide a sysctl machdep.cacheline_size to userland. It would be nice to
> have an MI API.
> >
> > On ARMv8, similarly, sometimes the kernel needs to advertise the wrong
> size.  A few big.LITTLE cores have 64-byte cache lines on one cluster and
> 32-byte on the other.  If you query the size from userspace while running
> on a 64-byte cluster, then issue the zero-cache-line instruction while
> migrated to the 32-byte cluster, you only clear half the size.  Linux works
> around this by trapping and emulating the instruction to query the cache
> size and always reporting the size for the smallest cache lines.  ARM tells
> people not to build systems like this, but it doesn???t always stop them.
> Trapping and emulating is much slower than just providing the information
> in a shared page, elf aux args vector, or even (often) a system call.
>
> Of course MD way is the best way to get such information, just because the
> meaning of the 'cache line size' exists only in context of the given CPU
> (micro)architecture.  For instance, on PowerPC and ARM you are often
> concerned
> with the granularity of the instruction cache flush, but also you might be
> concerned with the DMA, and these are different concepts of cache.
>
> Even on x86, you may care about alignment to avoid false sharing or
> about CLFLUSH granularity, and these can be different legitimately.
> Which one to report as 'cache line' ?
>
> And you cannot bail out with the max among all constants, because sometimes
> you really need the min size (for CLFLUSH), and sometime max size (for
> false sharing).
>
> >
> > To give another example, Linux provides a very cheap way for a userspace
> process to enquire which core it???s running on.  Some more recent
> high-performance mallocs use this to have a second-layer per-core cache
> after the per-thread cache for free blocks.  Unlike the per-thread cache,
> the per-core cache does need a lock, but it???s very unlikely to be
> contended (it will only be contended if either a thread is migrated in
> between checking and locking, so acquires the wrong CPU???s lock, or if a
> thread is preempted in the middle of middle of the very brief fill
> operation).  The author of the SuperMalloc paper tried doing this with
> CPUID and found that it was slower by a sufficient margin to almost
> entirely offset the benefits of the extra layer of caching.
>
> There, RDTSCP is the intended way to get cpu id in userspace, but the use
> of this instruction requires some minimal OS support.  It should be faster
> than CPUID, since it is not fully serializing.  We do not support it only
> because nobody asked so far.
>
> >
> > Just because userspace can get at the information directly from the
> hardware doesn???t mean that this is the most efficient or best way for
> userspace to get at it.
> >
> It depends, but single instruction (!) vs syscall comparision makes this
> discussion silly.
>
> > Oh, and some of these things are useful in portable code, so having to
> write some assembly for every target to get information that the kernel
> already knows is wasteful.
> >
> Required work is to provide the definitions of these interfaces, then they
> can be implemented in the best way for each architecture.  But nobody did
> that.
>
Initially I was "asking" to see if these facilities were available so that
I didn't have to go do some hack job
that would be brittle or reinvent the wheel.

My use case for getting the cache lines was to prevent false sharing.
I should've been more clear about that in my initial email but,
I didn't know all these other people were interested in this issue as well.

since I'd like the cache line sizes to prevent false sharing that's my use
case.

Does anyone else have any use cases that they would like to propose?

Once we have come to some agreement on what features they need,
then we can work out the interfaces and get the work done on implementation?

> _______________________________________________
> freebsd-current_at_freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"
>