Re: Strange ARC/Swap/CPU on yesterday's -CURRENT

On 2018-Mar-17, at 11:26 AM, Andriy Gapon <avg at FreeBSD.org> wrote:

> On 17/03/2018 18:51, Mark Millard wrote:
>> I'll  note that top was a -w that reports:
>> 
>>      -w     Display approximate swap usage for each process.
> 
> As far as I can tell, this option is quite broken.
> The "approximate swap usage" it reports is nowhere like it.

I have a hypothesis for part of what top is
counting in the process/thread SWAP column
that might not be what one would expect.

It appears to me that vnode-backed pages are
being re-classfied sometimes for inactive
processes, and this classification leads to
top classifying the pages as not-resident
but swapped (in that a "VN PAGER in" would
be required, in systat -vmstat terms).


Supporting details, if you care, otherwise
skip the below:

The hypothesis is from observing various hours
of over 20 hours of poudriere-devel "bulk -a"
activity, for which, at that point:

vm.stats.vm.v_swappgsout: 0
vm.stats.vm.v_swappgsin: 0
vm.stats.vm.v_swapout: 0
vm.stats.vm.v_swapin: 0

vm.stats.vm.v_vnodepgsout: 6996
vm.stats.vm.v_vnodepgsin: 32641833
vm.stats.vm.v_vnodeout: 1030
vm.stats.vm.v_vnodein: 4305027

Sometimes top showed lots of wait/select/pause
and such with positive SWAP and (mostly) 0K RES.
Most of the processes were in the "wait" STATE.
At other times, more like between 1 and 2 dozen
had positive SWAP.

There would be sudden large jumps in the number
of such processes. Then over time it would
decrease as the processes quit waiting (children
process trees finished).

The large jumps were not tied to Free becoming
small or anything else obvious from what I was
looking at. But the Free figure would increase
at that time. For example, I recently saw such
a large jump that was associated with Free
increasing from "90G" as shown in top.
(Much of the time there were between, say, 170
and 300 sleeping processes.)

The context was under Hyper-V with 29 logical
processors assigned to FreeBSD on a machine
with 16 cores/32 threads and 114 GiBytes of
RAM assigned to FreeBSD (of 128 GiBytes) and
256 GiBytes of swap-partition set up.
PARALLEL_JOBS allowed the 29 and ALLOW_MAKE_JOBS
was in use (allowing a potential for 29*29=841
or so running processes via poudriere bulk).



For reference: at 25 hours-in [idle] had 148.3H
(around 20% of the 29 threads * 25 H/thread) and
[bufdaemon] had 48.9H (around 6.7%). [kernel]
showed around 13.6H (817 min converted) and
[pagedaemon] showed around 1.7H (101 min
converted). Other processes had less TIME than
any of these.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)