Re: Processor cores not properly detected/activated?

On Thursday, May 29, 2014 12:53:54 pm Adrian Chadd wrote:
> On 28 May 2014 10:58, John Baldwin <jhb_at_freebsd.org> wrote:
> > On Wednesday, May 28, 2014 1:51:28 pm Adrian Chadd wrote:
> >> On 28 May 2014 06:56, John Baldwin <jhb_at_freebsd.org> wrote:
> >>
> >>
> >> > Userland cpusets only default to 128 (CPU_MAXSIZE in <sys/_cpuset.h>).
> >> > Changing MAXCPU to even 128 is unfortunately a potential KBI change since it
> >> > changes the size of 'cpuset_t'.  We can certainly bump these in HEAD for 11,
> >> > but we might not be able to MFC them without introducing ABI breakage.
> >> > (The cpuset APIs do allow the size of cpuset_t to change as the size is
> >> > encoded in the API calls, so there is that, it's more that if some public
> >> > structure embeds a cpuset_t in the kernel that we would have problems.  I
> >> > thought 'struct pcpu' did, but it does not.)
> >> >
> >> > Hmm, smp_rendezvous() accepts a cpuset_t as its first argument (and is a
> >> > public symbol used by kernel modules such as dtrace).  'struct rmlock' also
> >> > embeds a cpuset_t.  So, I think we can't bump cpuset_t without breaking
> >> > the KBI.  We can bump it in HEAD however.  (Note, if re_at_ signed off, we could
> >> > perhaps merge to 10, but we tend to be very hesitant about breaking the KBI.)
> >> > One thing we could do safely is bump the userland cpuset size to 256 in 10.
> >> > It's really only MAXCPU that is problematic.
> >> >
> >> > In particular, I propose we bump the userland cpuset_t size to 256 now (and
> >> > go ahead and merge that to 10).  In HEAD only we can bump MAXCPU for amd64
> >> > to 256.
> >>
> >> Since 11 is going to be around for a few years, can we experiment
> >> bumping it up to something compute-cluster-computer-sized just to get
> >> it over with? Something stupid, like 4096 or something?
> >
> > It costs wired memory to increase it for the kernel.  The userland set size
> > can be increased rather arbitrarily, so we don't need to make it but so large
> > as it is easy to bump later (even with a branch).
> 
> Well, what about making the API/KBI use cpuset_t pointers for things
> rather than including it as a bitmask? Do you think there'd be a
> noticable performance overhead for the bits where it's indirecting
> through a pointer to get to the bitmask data?

The wired memory is not due to cpuset_t.  The wired memory usage is due to things
that do 'struct foo foo_bits[MAXCPU]'.  The KBI issues I mentioned above are
'struct rmlock' (so now you want any rmlock users to malloc space, or you
want rmlock_init() call malloc?  (that seems like a bad idea)).  The other one
is smp_rendezvous.  Plus, it's not just a pointer, you really need a (pointer,
size_t) tuple similar to what cpuset_getaffinity(), etc. use.

-- 
John Baldwin