Re: ARC "pressured out", how to control/stabilize ? (reformatted to text/plain)

on 15/01/2014 12:28 Vitalij Satanivskij said the following:
> Dear Andriy and FreeBSD community,
> 
> Andriy Gapon wrote:
> AG> on 14/01/2014 07:27 Vladimir Sharun said the following:
> AG> > Dear Andriy and FreeBSD community,
> AG> > 
> AG> >> I am not sure if the buffers are leaked somehow or if they are actually in use.
> AG> >> It's one of the very few places where data buffers are allocated without
> AG> >> charging ARC.  In all other places it's quite easy to match allocations and
> AG> >> deallocations.  But in L2ARC it is not obvious that all buffers get freed or
> AG> >> when that happens.
> AG> > 
> AG> > After one week under load I think we figure out the cause: it's L2ARC. 
> AG> > Here's the top's header for 7d17h of the runtime:
> AG> > 
> AG> > last pid: 46409;  load averages:  0.37,  0.62,  0.70 up 7+17:14:01  07:24:10
> AG> > 173 processes: 1 running, 171 sleeping, 1 zombie
> AG> > CPU:  2.0% user,  0.0% nice,  3.5% system,  0.4% interrupt, 94.2% idle
> AG> > Mem: 8714M Active, 14G Inact, 96G Wired, 1929M Cache, 3309M Buf, 3542M Free
> AG> > ARC: 85G Total, 2558M MFU, 77G MRU, 28M Anon, 1446M Header, 4802M Other
> AG> > 
> AG> > ARC related tunables:
> AG> > 
> AG> > vm.kmem_size="110G"
> AG> > vfs.zfs.arc_max="90G"
> AG> > vfs.zfs.arc_min="42G"
> AG> > 
> AG> > For more than 7 days of hard runtime the picture clearly shows: 
> AG> > Wired minus ARC = 11..12Gb, ARC grow and shrinks in 80-87Gb range and the
> AG> > system runs just fine.
> AG> > 
> AG> > So what shall we do with L2ARC leakage ?
> AG> 
> AG> 
> AG> Could you please try this patch
> AG> http://cr.illumos.org/~webrev/skiselkov/3995/illumos-gate.patch ?
> AG> 
> 
> While applying path to curent version of arc.c (r260622) I'm found next truble with compilation 
> 
> olaris/uts/common/fs/zfs/arc.c -o arc.o
> /usr/src/cddl/lib/libzpool/../../../sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:4628:18: error: use of
>       undeclared identifier 'abl2'
>                         trim_map_free(abl2->b_dev->l2ad_vdev, abl2->b_daddr,
>                                       ^
> 1 error generated.
> *** Error code 1
> 
> 
> the code is - 
> 
>                 if (zio->io_error != 0) {
>                         /*
>                          * Error - drop L2ARC entry.
>                          */
>                         list_remove(buflist, ab);
>                         ARCSTAT_INCR(arcstat_l2_asize, -l2hdr->b_asize);
>                         ab->b_l2hdr = NULL;
>                         trim_map_free(abl2->b_dev->l2ad_vdev, abl2->b_daddr,
>                             ab->b_size, 0);
>                         kmem_free(l2hdr, sizeof (l2arc_buf_hdr_t));
>                         ARCSTAT_INCR(arcstat_l2_size, -ab->b_size);
>                 }
> 
> 
> Looks like it's part is freebsd specific changes.
> Can somebody help with this part of code ?
> 

The first hunk of the patch is renaming of abl2 to l2hdr.

-- 
Andriy Gapon