Re: [PATCH] Convert the VFS cache lock to an rmlock

From: Mateusz Guzik <mjguzik_at_gmail.com>
Date: Fri, 13 Mar 2015 06:32:03 +0100
On Thu, Mar 12, 2015 at 06:13:00PM -0500, Alan Cox wrote:
> Below is partial results from a profile of a parallel (-j7) "buildworld" on
> a 6-core machine that I did after the introduction of pmap_advise, so this
> is not a new profile.  The results are sorted by total waiting time and
> only the top 20 entries are listed.
> 

Well, I ran stuff on lynx2 in the zoo on fresh -head with debugging
disabled (MALLOC_PRODUCTION included) and got quite different results.

The machine is  Intel(R) Xeon(R) CPU E5-2680 v2 _at_ 2.80GHz
 2 package(s) x 10 core(s) x 2 SMT threads

32GB of ram

Stuff was built in a chroot with world hosted on zfs.

>      max  wait_max       total  wait_total       count    avg wait_avg
> cnt_hold cnt_lock name
> 
>     1027    208500    16292932  1658585700     5297163      3    313  0
> 3313855 kern/vfs_cache.c:629 (rw:Name Cache)
> 
>   208564    186514 19080891106  1129189627   355575930     53      3  0
> 1323051 kern/vfs_subr.c:2099 (lockmgr:ufs)
> 
>   169241    148057   193721142   419075449    13819553     14     30  0
> 110089 kern/vfs_subr.c:2210 (lockmgr:ufs)
> 
>   187092    191775  1923061952   257319238   328416784      5      0  0
> 5106537 kern/vfs_cache.c:488 (rw:Name Cache)
> 

make -j 12 buildworld on freshly booted system (i.e. the most namecache insertions):

      32       292     3042019    33400306     8419725      0      3  0 2578026 kern/sys_pipe.c:1438 (sleep mutex:pipe mutex)
  170608    152572   642385744    27054977   202605015      3      0  0 1306662 kern/vfs_subr.c:2176 (lockmgr:zfs)
      66       198    45170221    22523597   161976016      0      0  0 19988525 vm/vm_page.c:1502 (sleep mutex:vm page free queue)
      45       413    17804028    20770896   160786478      0      0  0 19368394 vm/vm_page.c:2294 (sleep mutex:vm page free queue)
      32       625    19406548     8414459   142554059      0      0  0 11198547 vm/vm_page.c:2053 (sleep mutex:vm active pagequeue)
      35      1704    19560396     7867435   142655646      0      0  0 9641161 vm/vm_page.c:2097 (sleep mutex:vm active pagequeue)
      14   6675994       27451     6677152       53550      0    124  0   2394 kern/sched_ule.c:2630 (spin mutex:sched lock 23)
    2121       879    19982319     4157880     7753007      2      0  0 235477 vm/vm_fault.c:785 (rw:vm object)
   27715      1104     9922805     3339829    12614622      0      0  0  83840 vm/vm_map.c:2883 (rw:vm object)
       6   2240335       26594     2833529       55057      0     51  0   2643 kern/sched_ule.c:2630 (spin mutex:sched lock 17)
      31     22617     1424889     2768798      368786      3      7  0  11555 kern/kern_exec.c:590 (lockmgr:zfs)
       7   2027019       26247     2218701       53980      0     41  0   2432 kern/sched_ule.c:2630 (spin mutex:sched lock 5)
   57942    153184    41616375     2120917      368786    112      5  0   9438 kern/imgact_elf.c:829 (lockmgr:zfs)
     184       557    65168745     1715154   214930217      0      0  0 2104013 kern/vfs_cache.c:487 (rw:Name Cache)
^^^^ name cache only here
       3   1695608       26302     1696719       56150      0     30  0   2377 kern/sched_ule.c:2630 (spin mutex:sched lock 18)
      52       176    49658348     1606545   199234071      0      0  0 2212598 kern/vfs_cache.c:668 (sleep mutex:vnode interlock)
       6   1497134       26337     1583199       55416      0     28  0   2096 kern/sched_ule.c:2630 (spin mutex:sched lock 13)
    1705      2155    55312677     1519894   142655701      0      0  0 435090 vm/vm_fault.c:997 (sleep mutex:vm page)
      14       721      187832     1449400     2126043      0      0  0  28314 vm/vm_object.c:646 (rw:vm object)
      74        62    31785614     1342727   268853124      0      0  0 2235545 kern/vfs_subr.c:2254 (sleep mutex:vnode interlock)

So even despite the need for a lot of insertions name cache contention was not a
big concern.

Here is another buildworld after clearing /usr/obj and resetting stats (no reboot,
so the cache was populated):

      31       378     3827573    40116363     8544224      0      4  0 2602464 kern/sys_pipe.c:1438 (sleep mutex:pipe mutex)
      53       680    45790806    26978449   161004693      0      0  0 21077331 vm/vm_page.c:1502 (sleep mutex:vm page free queue)
      39       210    18513457    25286194   160721062      0      0  0 20946513 vm/vm_page.c:2294 (sleep mutex:vm page free queue)
   19806     19377   596036045    19086474   202605527      2      0  0 1361883 kern/vfs_subr.c:2176 (lockmgr:zfs)
      40       810    19593696     9458254   142544401      0      0  0 11659059 vm/vm_page.c:2053 (sleep mutex:vm active pagequeue)
      45      1713    19955926     8883884   142638570      0      0  0 10061154 vm/vm_page.c:2097 (sleep mutex:vm active pagequeue)
      15   4702161       28765     4715991       59859      0     78  0   2659 kern/sched_ule.c:2630 (spin mutex:sched lock 6)
    1838      1213    20189264     4246013     7751227      2      0  0 243511 vm/vm_fault.c:785 (rw:vm object)
   34942       782    10815453     3461181    12611561      0      0  0  87884 vm/vm_map.c:2883 (rw:vm object)
       7   2111512       27390     3164572       55775      0     56  0   2239 kern/sched_ule.c:2630 (spin mutex:sched lock 7)
      18      2503     1417189     2849233      368785      3      7  0  12099 kern/kern_exec.c:590 (lockmgr:zfs)
      52       898    66378192     1861837   214861582      0      0  0 2221482 kern/vfs_cache.c:487 (rw:Name Cache)
      16        52    49359798     1685568   199202365      0      0  0 2288836 kern/vfs_cache.c:668 (sleep mutex:vnode interlock)
      13       811      190617     1527468     2125719      0      0  0  30154 vm/vm_object.c:646 (rw:vm object)
      38        39    31672997     1393102   268812945      0      0  0 2304916 kern/vfs_subr.c:2254 (sleep mutex:vnode interlock)
    1714      2111    56782095     1303511   142638594      0      0  0 199781 vm/vm_fault.c:997 (sleep mutex:vm page)
      15    765633       28820     1220541       59670      0     20  0   2805 kern/sched_ule.c:2630 (spin mutex:sched lock 8)
     177       143    59407392     1213817    58267983      1      0  0 377555 amd64/amd64/pmap.c:5362 (rw:pmap pv list)
      37        21    28518097     1199499   291370530      0      0  0 1372317 kern/subr_sleepqueue.c:251 (spin mutex:sleepq chain)
      15    809971       29102     1103478       59356      0     18  0   2737 kern/sched_ule.c:2630 (spin mutex:sched lock 19)

So, it may be somewthing is wrong with my test environment, but as it is I do
not expect namecache lock contention to have significant impact on buildworld/kernel.

-- 
Mateusz Guzik <mjguzik gmail.com>
Received on Fri Mar 13 2015 - 04:32:10 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:56 UTC