Re: how to recycle Inact memory more aggressively?

rsync... see bottom posting

On Tue, 15 Mar 2016 07:43:46 +0100, olli hauer <ohauer_at_gmx.de> wrote:

> On 2016-03-14 15:19, Ian Lepore wrote:
> > On Sun, 2016-03-13 at 19:08 -0700, Adrian Chadd wrote:
> >> On 13 March 2016 at 18:51, Mark Johnston <markj_at_freebsd.org> wrote:
> >>> On Sun, Mar 13, 2016 at 06:33:46PM -0700, Adrian Chadd wrote:
> >>>> Hi,
> >>>>
> >>>> I can reproduce this by doing a mkimage on a large destination
> >>>> file
> >>>> image. it looks like it causes all the desktop processes to get
> >>>> paged
> >>>> out whilst it's doing so, and then the whole UI freezes until it
> >>>> catches up.
> >>>
> >>> mkimg(1) maps the destination file with MAP_NOSYNC, so if it's
> >>> larger
> >>> than RAM, I think it'll basically force the pagedaemon to write out
> >>> the
> >>> image as it tries to reclaim pages from the inactive queue. This
> >>> can
> >>> cause stalls if the pagedaemon blocks waiting for some I/O to
> >>> complete.
> >>> The user/alc/PQ_LAUNDRY branch helps alleviate this problem by
> >>> using a
> >>> different thread to launder dirty pages. I use mkimg on various
> >>> desktop
> >>> machines to build bhyve images and have noticed the problem you're
> >>> describing; PQ_LAUNDRY helps quite a bit in that case. But I don't
> >>> know
> >>> why this would be a new problem.
> >>>
> >>
> >> That's why I'm confused. I just know that it didn't used to cause the
> >> whole UI to hang due to paging.
> >>
> > 
> > I've been noticing this too.  This machine runs 10-stable and this use
> > of the swap began happening recently when I updated from 10-stable
> > around the 10.2 release time to 10-stable right about when the 10.3
> > code freeze began.
> > 
> > In my case I have no zfs anything here.  I noticed the problem bigtime
> > yesterday when rsync was syncing a ufs filesystem of about 500GB from
> > one disk to another (probably 70-80 GB actually needed copying).  My
> > desktop apps were noticibly unresponsive when I activated a window that
> > had been idle for a while (like it would take a couple seconds for the
> > app to begin responding).  I could see lots of swap In happening in top
> > during this unresponsiveness, and noticible amounts of Out activity
> > when nothing was happening except the rsync.
> > 
> > This is amd64, 12GB ram, 16GB swap, a tmpfs had about 400MB in it at
> > the time.  Prior to the update around the 10.3 freeze, this machine
> > would never touch the swap no matter what workload I threw at it (this
> > rsync stuff happens every day, it's the usual workload).
> > 
> 
> I'm not sure if it is the same problem, or port related.
> 
> On two systems without zfs but with many files e.g. svn servers I see now
> from time to time they are running out of swap.
> 
>  kernel: swap_pager_getswapspace(9): failed
>  kernel: swap_pager_getswapspace(16): failed
>  ...
> 
> It also happened on one system during the nightly periodic tasks holding
> only millions of backup files.
> 
> $ freebsd-version -ku
>   10.2-RELEASE-p9
>   10.2-RELEASE-p13
> 
> 
> _______________________________________________
> freebsd-current_at_freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"


Just a point I've bought up elsewhere...
I've, if I recall, wrecked several filesystems (although EIDE) using rsync at the normal bus rate, and sometimes
thumbdrives with whatever filesystem type on them.

I settled on --bwlimit=1500,  max for unattended  rsync usage and almost every day
use --bwlimit=700.

The latter enables several resource-intensive processes ( music, classical music videos, svn, pkg, browsing, etc) to
proceed apace concurrently on the desktop (SATA not EIDE) with nary a hang nor slowdown.

If I recall, the usual speed is 10000 so that is less than ten percent, if I recall, of the usual speed.

YMMV.

J.

PS as an afterthough, it would be useful if that were more prominent on the man page somewhere or even
in the port's pkg-message or pkg-description.  
The SATA more robust than EIDE on FreeBSD that I've come across, though I prefer not to hint at because I
believe it to be the fault of EIDE firmware rather than FreeBSD code. FWIW.