Re: SCHED_ULE should not be the default

On Mon Dec 19 11, Nathan Whitehorn wrote:
> On 12/18/11 04:34, Adrian Chadd wrote:
> >The trouble is that there's lots of anecdotal evidence, but noone's
> >really gone digging deep into _their_ example of why it's broken. The
> >developers who know this stuff don't see anything wrong. That hints to
> >me it may be something a little more creepy - as an example, the
> >interplay between netisr/swi/taskqueue/callbacks and such. It may be
> >that something is being starved that isn't obviously obvious. It's
> >just a stab in the dark, but it sounds somewhat plausible based on
> >what I've seen ULE do in my network throughput hacking.
> >
> >I applaud reppie for trying to make it as easy as possible for people
> >to use KTR to provide scheduler traces for him to go digging with, so
> >please, if you have these issues and you can absolutely reproduce
> >them, please follow his instructions and work with him to get him what
> >he needs.
> 
> The thing I've seen is that ULE is substantially more enthusiastic about 
> migrating processes between cores than 4BSD. Often, this is a good 
> thing, but can increase the rate of cache misses, hurting performance 
> for cache-bound processes (I see this particularly in HPC-type 
> scientific workloads). It might be interesting to add some kind of 
> tunable here.

does r228718 have any impact regarding this behaviour?

cheers.
alex

> 
> Another more interesting and slightly longer-term possibility if someone 
> wants a project would be to integrate scheduling decisions with hwpmc 
> counters, to accumulate statistics on cache hits at each context switch 
> and preferentially keep processes with a high hits/misses ratio on the 
> same thread/cache domain relative to processes with a low one.
> -Nathan
> 
> P.S. The other thing that could be very interesting from a research and 
> scheduling standpoint would be to integrate heterogeneous SMP support 
> into the operating system, with a FreeBSD-4 "Application Processor" 
> syscall model. We seem to be going down the road where GPGPU computing 
> has MMUs, timer interrupts, IPIs, etc. (the next AMD Fusions, IBM Cell). 
> This is something that no operating system currently supports well, and 
> would be a place for BSD to shine. If anyone has a free graduate student...