RE: stable/13, vm page counts do not add up

> -----Original Message-----
> From: owner-freebsd-current_at_freebsd.org <owner-freebsd-
> current_at_freebsd.org> On Behalf Of Mark Johnston
> Sent: Wednesday, April 7, 2021 10:57 PM
> To: Andriy Gapon <avg_at_freebsd.org>
> Cc: freebsd-stable List <stable_at_freebsd.org>; FreeBSD Current
> <current_at_freebsd.org>
> Subject: Re: stable/13, vm page counts do not add up
> 
> On Wed, Apr 07, 2021 at 11:22:41PM +0300, Andriy Gapon wrote:
> > On 07/04/2021 22:54, Mark Johnston wrote:
> > > On Wed, Apr 07, 2021 at 10:42:57PM +0300, Andriy Gapon wrote:
> > >>
> > >> I regularly see that the top's memory line does not add up (and by a lot).
> > >> That can be seen with vm.stats as well.
> > >>
> > >> For example:
> > >> $ sysctl vm.stats | fgrep count
> > >> vm.stats.vm.v_cache_count: 0
> > >> vm.stats.vm.v_user_wire_count: 3231
> > >> vm.stats.vm.v_laundry_count: 262058
> > >> vm.stats.vm.v_inactive_count: 3054178
> > >> vm.stats.vm.v_active_count: 621131
> > >> vm.stats.vm.v_wire_count: 1871176
> > >> vm.stats.vm.v_free_count: 187777
> > >> vm.stats.vm.v_page_count: 8134982
> > >>
> > >> $ bc
> > >>>>> 187777 + 1871176 + 621131 + 3054178 + 262058
> > >> 5996320
> > >>>>> 8134982 - 5996320
> > >> 2138662
> > >>
> > >> As you can see, it's not a small number of pages either.
> > >> Approximately 2 million pages, 8 gigabytes or 25% of the whole memory
> on this
> > >> system.
> > >>
> > >> This is 47c00a9835926e96, 13.0-STABLE amd64.
> > >> I do not think that I saw anything like that when I used (much) older
> FreeBSD.
> > >
> > > One relevant change is that vm_page_wire() no longer removes pages
> from
> > > LRU queues, so the count of pages in the queues can include wired
> pages.
> > > If the page daemon runs, it will dequeue any wired pages that are
> > > encountered.
> >
> > Maybe I misunderstand how that works, but I would expect that the sum
> of all
> > counters could be greater than v_page_count at times.  But in my case it's
> less.
> 
> I misread, sorry.  You're right, what I described would cause double
> counting.
> 
> I don't know what might be causing it then.  It could be a page leak.
> The kernel allocates wired pages without adjusting the v_wire_count
> counter in some cases, but the ones I know about happen at boot and
> should not account for such a large disparity.  I do not see it on a few
> systems that I have access to.
> 
> > > This was done to reduce queue lock contention, operations like
> > > sendfile() which transiently wire pages would otherwise trigger two
> > > queue operations per page.  Now that queue operations are batched this
> > > might not be as important.
> > >
> > > We could perhaps add a new flavour of vm_page_wire() which is not lazy
> > > and would be suited for e.g., the buffer cache.  What is the primary
> > > source of wired pages in this case?
> >
> > It should be ZFS, I guess.
> >
> > --
> > Andriy Gapon
> _______________________________________________
> freebsd-current_at_freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-
> unsubscribe_at_freebsd.org"

I see kernel memory disappearing, when enabling ktls:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253281

Last test done with 13.0-RC1. 

I'm a bit at a loss how to debug this further.

Regards 

Juergen Weiss

Juergen Weiss      |
weiss_at_uni-mainz.de |