Re: CURRENT slow and shaky network stability

From: Cy Schubert <Cy.Schubert_at_komquats.com>
Date: Sat, 02 Apr 2016 00:21:24 -0700
In message <56F6C6B0.6010103_at_protected-networks.net>, Michael Butler writes:
> -current is not great for interactive use at all. The strategy of
> pre-emptively dropping idle processes to swap is hurting .. big time.
> 
> Compare inactive memory to swap in this example ..
> 
> 110 processes: 1 running, 108 sleeping, 1 zombie
> CPU:  1.2% user,  0.0% nice,  4.3% system,  0.0% interrupt, 94.5% idle
> Mem: 474M Active, 1609M Inact, 764M Wired, 281M Buf, 119M Free
> Swap: 4096M Total, 917M Used, 3178M Free, 22% Inuse
> 
>   PID USERNAME       THR PRI NICE   SIZE    RES STATE   C   TIME    WCPU
> COMMAND
>  1819 imb              1  28    0   213M 11284K select  1 147:44   5.97%
> gkrellm
> 59238 imb             43  20    0   980M   424M select  0  10:07   1.92%
> firefox
> 
>  .. it shouldn't start randomly swapping out processes because they're
> used infrequently when there's more than enough RAM to spare ..

Inactive memory will after time be used to "top up" the free memory pool.

> 
> It also shows up when trying to reboot .. on all of my gear, 90 seconds
> of "fail-safe" time-out is no longer enough when a good proportion of
> daemons have been dropped onto swap and must be brought back in to flush
> their data segments :-(

What does vmstat 5 display? A high scan rate is indicative of memory in short supply.

My laptop has 6 GB RAM of which I've allocated 2.5 GB for ARC. Top shows that 3.6 GB 
are wired (not pagable) leaving 2.4 GB available for apps. The laptop is in the 
middle of a four thread buildworld. It's using 1.8 MB swap so far. ARC is 2560 MB. 
UFS cache is only 49 MB at the moment but will balloon to 603 MB during installworld. 
When it does my swap grows to 100-150 MB. (The reason is the large 2.5 GB ARC *and* a 
large 603 MB UFS cache.)

Notice vmstat output below. On line 8 of my vmstat output you scan rate jump to 10K. 
Two pages per second are paged out. This is due to the free memory pool (in line 7) 
dropping to 49 MB, so it freed up a few pages by paging out. Notice that page 
reclaims (re) is high at times. These pages were scheduled to be paged out but were 
used before they were. This indicates that my laptop is is running pretty close to 
the line between paging a lot and not paging at all.

slippy$ vmstat 5
 procs      memory      page                    disks     faults         cpu
 r b w     avm    fre   flt  re  pi  po    fr  sr ad0 da0   in   sy   cs us sy id
 4 0 0   3413M   208M 12039  11   9   0 12804 170   0   0  533 7865 2722 59  3 38
 4 0 0   3260M   376M 18550   0   0   0 27436 386   9   8  576 23029 30705 94  6  0
 4 1 0   3432M   171M 25345   0   6   0 15340 360  10  12  530 2524 1362 97  3  0
 4 0 0   3395M   208M 20904   0   0   0 22995 395  12  11  517 5427 1142 97  3  0
 4 0 0   3695M    53M 20102   0   0   0 12482 473  17  10  517 1383 1244 98  2  0
 4 0 0   3404M   371M 22996  14  10   0 39691 4557  14   8  503 8540 1813 96  3  1
 4 1 0   3673M    49M 22398 441  22   0  6778 429  10  13  543 3034 1609 97  3  0
 4 0 0   3396M   439M 19522  26   3   2 33901 10137  11  15  545 5617 1686 97  3  0
 4 0 0   3489M   412M 26636   0   0   0 25710 393  10  12  531 5287 1450 95  3  2
 4 0 0   3558M   337M 23364 329  13   0 20051 410  11  15  561 6052 1702 96  3  0
 4 0 0   3492M   335M 18244   0   3   0 18550 444  14   7  512 5140 2087 98  2  0
 4 0 0   3412M   404M 21765   0   0   0 25611 388   7  12  533 7873 1394 97  3  0
 5 0 0   3604M   189M 19044   0   0   0  8404 505   7  10  644 63940 90591 93  6  1
 4 0 0   3533M   363M 13079 423  17   0 22327 464  11   8  501 7960 4194 94  3  3
 4 0 0   3222M   616M 20822 218  17   0 34180 294  11  13  550 5602 1850 95  4  1
 4 0 0   3307M   542M 19639  32   3   0 15940 345  13  10  516 2589 1505 96  3  1
 4 0 0   3320M   527M 19656   0   1   0 19191 397  14   8  514 1886 1257 97  3  0
 4 0 0   3295M   605M 21676 910  35   0 25978 356  14  12  533 3039 1490 95  4  0

Page outs is the first place to look. If no page outs, page reclaims will tell you 
your system may be borderline. A high scan rate says that your working set size is 
large enough to put some  pressure on VM. Ideally in this case I should add memory 
but since I'm running this close to the line (I chose my ARC maximum well) I'll just 
save my money instead. Also, FreeBSD is doing exactly what it should in this scenario.

Top is a good tool but it doesn't tell the whole picture. Run vmstat to get a better 
picture of your memory use.

Following this thread throughout the day (on my cellphone), I'm not convinced this is 
a FreeBSD O/S problem. Check your apps. What are their working set sizes. Do your 
apps have a small or large locality of reference? Remember, O/S tuning is a matter of 
robbing Peter to pay Paul. Reducing the resources used by applications will pay back 
bigger dividends.

Hope this helps.


-- 
Cheers,
Cy Schubert <Cy.Schubert_at_komquats.com> or <Cy.Schubert_at_cschubert.com>
FreeBSD UNIX:  <cy_at_FreeBSD.org>   Web:  http://www.FreeBSD.org

	The need of the many outweighs the greed of the few.
Received on Sat Apr 02 2016 - 05:21:35 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:03 UTC