ULE vs. 4BSD scheduler benchmarks

[current_at_ bcc'ed to get a wider audience, please discuss on performance_at_]

Hi,

in recent times i saw a lot of threads where it was suggested people
should switch from the ULE to the 4BSD scheduler. That got me thinking
and i decided to run a few benchmarks. I looked through all the stuff
Kris and Jeff did a few years ago and tried to follow their example. The
main motivation is however that we (Attilio Rao and I) want to set a
baseline for future reference, mainly for the work that's going on in
the vmcontention branch right now, that is the reason why all tests were
run on head_at_r229659. All debugging was disabled (WITNESS and friends for
the kernel and MALLOC_PRODUCTION=yes for libc).

For now i ran 3 different things. MySQL/sysbench, PostgreSQL/pgbench and
pbzip2.

All software was installed from ports with the default system gcc (gcc
version 4.2.1 20070831 patched [FreeBSD]), with the exception of
PostgreSQL. I created new postgres92-{server,client} ports with a
snapshot of PostgreSQL 9.2dev from 16.01.2012, as a lot of scalability
work was done in PostgreSQL 9.2.

MySQL version 5.5.20
sysbench version 0.4.12
PostgreSQL/pgbench version 9.2dev
PBZIP2 version v1.1.6

The machine these test were run on is a 2x4 core Xeon L5310  _at_ 1.60GHz
with 4GB RAM. Here is the complete topology:

kern.sched.topology_spec: <groups>
 <group level="1" cache-level="0">
  <cpu count="8" mask="ff">0, 1, 2, 3, 4, 5, 6, 7</cpu>
  <children>
   <group level="2" cache-level="2">
    <cpu count="4" mask="f">0, 1, 2, 3</cpu>
   </group>
   <group level="2" cache-level="2">
    <cpu count="4" mask="f0">4, 5, 6, 7</cpu>
   </group>
  </children>
 </group>
</groups>

The database benchmarks were all run with a work set that fit into the
configured database memory, so after the warmup phase no disk io was
involved. sysbench was run with 1 million rows, innodb was the engine we
used as Kris work already showed that it scales much better than myisam
(also innodb is the default in MySQL's 5.5 branch). Pgbench was run
using a scaling factor of 100. The connection to the databases was using
a unix socket, also only read only tests were run.

The input and output files for the pbzip2 test were on tmpfs.

The results are available in this Google docs spreadsheet, if you scroll
down there are also some nice graphs.

https://docs.google.com/spreadsheet/ccc?key=0Ai0N1xDe3uNAdDRxcVFiYjNMSnJWOTZhUWVWWlBlemc

Over time i will add more benchmarks to the doc (i.e nginx/php-fpm and
so on). I tried to run some nginx benchmarks, but those are limited by
netisr, as i did not find a web server benchmark tool which can use unix
sockets, any suggestions welcome.

The conclusion right now seems to be that ULE is faster for database
workload, but for strongly CPU-bound workloads 4BSD can be a better
choice. I can provide KTR traces and/or schedgraph output for cases
where 4BSD is better than ULE.

I want to thank Sean Bruno and Yahoo for setting up / providing the
machines to run these test on, and Attilio for suggestions and his
general helpfulness.

Florian