Re: Strange ARC/Swap/CPU on yesterday's -CURRENT

Am 05.03.18 um 21:39 schrieb Larry Rosenman:
> Upgraded to:
> 
> FreeBSD borg.lerctr.org 12.0-CURRENT FreeBSD 12.0-CURRENT #11 r330385: Sun Mar  4 12:48:52 CST 2018     root_at_borg.lerctr.org:/usr/obj/usr/src/amd64.amd64/sys/VT-LER  amd64
> +1200060 1200060
> 
> Yesterday, and I'm seeing really strange slowness, ARC use, and SWAP use and swapping.
> 
> See http://www.lerctr.org/~ler/FreeBSD/Swapuse.png
> 
> Ideas?

I'm seeing the same, and currently work around this with a reasonably limited
vfs.zfs.arc_max.

Without such a limit I see (on a system with 24 GB RAM):

CPU:  0.3% user,  0.0% nice,  0.9% system,  0.1% interrupt, 98.8% idle
Mem: 14M Active, 1228K Inact, 32K Laundry, 23G Wired, 376M Free
ARC: 19G Total, 3935M MFU, 14G MRU, 82M Anon, 223M Header, 876M Other
     18G Compressed, 36G Uncompressed, 2.02:1 Ratio
Swap: 24G Total, 888M Used, 23G Free, 3% Inuse, 8892K In, 5136K Out

sysctl vfs.zfs.arc_max=15988656640 results in:

Mem: 129M Active, 72M Inact, 36K Laundry, 18G Wired, 5149M Free
ARC: 15G Total, 3997M MFU, 10G MRU, 40M Anon, 205M Header, 877M Other
     13G Compressed, 28G Uncompressed, 2.08:1 Ratio
Swap: 24G Total, 796M Used, 23G Free, 3% Inuse, 16K In

The system was mostly idle at both times, just some Samba traffic and
mail being checked by spamassassin. And I noticed it (this time) when
the spamassassin processes were aborted due to a time limit.

I think that this problem must have been introduced in the last few
weeks, but cannot give a better estimate (do not reboot that often).

But I had already applied the arc_max setting a week ago (and had not
put it in sysctl.conf in the hope that the ARC growth was a temporary
problem in the ZFS code, soon to be fixed ...).

Regards, STefan