Re: Strange ARC/Swap/CPU on yesterday's -CURRENT

From: Larry Rosenman <ler_at_lerctr.org>
Date: Tue, 03 Apr 2018 21:46:21 -0500
Thanks for that hint.  I thought that arc_max was a tunable, but now knowing it's sysctl, that helps a lot.


-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 214-642-9640                 E-Mail: ler_at_lerctr.org
US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106
On 4/3/18, 8:54 PM, "Cy Schubert" <owner-freebsd-current_at_freebsd.org on behalf of Cy.Schubert_at_cschubert.com> wrote:

    Try arbitrarily reducing arc_max through sysctl. ARC is immediately reduced and free memory increased however wired pages remains the same.
    
    ---
    Sent using a tiny phone keyboard.
    Apologies for any typos and autocorrect.
    Also, this old phone only supports top post. Apologies.
    
    Cy Schubert
    <Cy.Schubert_at_cschubert.com> or <cy_at_freebsd.org>
    The need of the many outweighs the greed of the few.
    ---
    
    -----Original Message-----
    From: Larry Rosenman
    Sent: 03/04/2018 19:27
    To: Cy Schubert; Bryan Drewery; Peter Jeremy; Jeff Roberson
    Cc: FreeBSD current; Andriy Gapon
    Subject: Re: Strange ARC/Swap/CPU on yesterday's -CURRENT
    
    When my full backups run (1st Sunday -> Monday of the month) the box becomes unusable after 5-10 hours of that backup with LOTS of SWAP usage
    And ARC using 100+G. 
    
    Is anyone looking into this?
    
    
    
    -- 
    Larry Rosenman                     http://www.lerctr.org/~ler
    Phone: +1 214-642-9640                 E-Mail: ler_at_lerctr.org
    US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106
    On 4/3/18, 8:24 PM, "Cy Schubert" <owner-freebsd-current_at_freebsd.org on behalf of Cy.Schubert_at_cschubert.com> wrote:
    
        +1
        
        However under certain circumstances it will release some memory. To reproduce, when bsdtar unpacks some tarballs (when building certain ports) tar will use 12 GB or  more  forcing ARC to release memory.
        
        BTW, I haven't stopped to grok whether the bsdtar issue is local to me or another problem yet.
        
        ---
        Sent using a tiny phone keyboard.
        Apologies for any typos and autocorrect.
        Also, this old phone only supports top post. Apologies.
        
        Cy Schubert
        <Cy.Schubert_at_cschubert.com> or <cy_at_freebsd.org>
        The need of the many outweighs the greed of the few.
        ---
        
        -----Original Message-----
        From: Bryan Drewery
        Sent: 23/03/2018 17:23
        To: Peter Jeremy; Jeff Roberson
        Cc: FreeBSD current; Andriy Gapon
        Subject: Re: Strange ARC/Swap/CPU on yesterday's -CURRENT
        
        On 3/20/2018 12:07 AM, Peter Jeremy wrote:
        > 
        > On 2018-Mar-11 10:43:58 -1000, Jeff Roberson <jroberson_at_jroberson.net> wrote:
        >> Also, if you could try going back to r328953 or r326346 and let me know if 
        >> the problem exists in either.  That would be very helpful.  If anyone is 
        >> willing to debug this with me contact me directly and I will send some 
        >> test patches or debugging info after you have done the above steps.
        > 
        > I ran into this on 11-stable and tracked it to r326619 (MFC of r325851).
        > I initially got around the problem by reverting that commit but either
        > it or something very similar is still present in 11-stable r331053.
        > 
        > I've seen it in my main server (32GB RAM) but haven't managed to reproduce
        > it in smaller VBox guests - one difficulty I faced was artificially filling
        > ARC.
        > 
        
        Looking at the ARC change you referred to from r325851
        https://reviews.freebsd.org/D12163, I am convinced that ARC backpressure
        is completely broken. On my 78GB RAM system with ARC limited to 40GB and
        doing a poudriere build of all LLVM and GCC packages at once in tmpfs I
        can get swap up near 50GB and yet the ARC remains at 40GB through it
        all.  It's always been slow to give up memory for package builds but it
        really seems broken right now.
        
        -- 
        Regards,
        Bryan Drewery
        
        _______________________________________________
        freebsd-current_at_freebsd.org mailing list
        https://lists.freebsd.org/mailman/listinfo/freebsd-current
        To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"
        
    
    
    _______________________________________________
    freebsd-current_at_freebsd.org mailing list
    https://lists.freebsd.org/mailman/listinfo/freebsd-current
    To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"
    
Received on Wed Apr 04 2018 - 00:46:24 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:15 UTC