Re: vmstat's entries type

On Sun, 30 Jul 2006, Michal Mertl wrote:

> Michal Mertl wrote:
>> Paul Allen wrote:
>>> Surely all you need to do is a cheap crit_enter,crit_exit
>>> while updating the 64-bit per cpu counters.  and on
>>> a 64-bit arch you skip the crit_enter,crit_exit.
>>
>> Critical_enter/exit seem to be quite lightweight (single read/modify/write 
>> of a variable).
>
> One more question. Why do you say that crit_* can be avoided on 64-bit arch? 
> If the reason was that "increment of a 64 bit number is one operation there" 
> it probably is not true - as somebody already stated, some instruction sets 
> don't allow atomic increment of a memory location.

The UMA zone allocator and malloc allocator both use critical sections to 
protect their per-cpu statistics and this works quite well in practice. 
However, critical sections are also significantly not-free compared to a 
simple integer increment.  The argument for using 64-bit arithmetic without 
synchronization is that a small amount of occasional inaccuracy could be 
tolerated (a 64-bit increment being lost due to a race, for example), but that 
large amounts associated with 64-bit arithmetic using 32-bit operations, might 
not be (a 32-bit wrap being counted 0 times or 2 times).

If statistics are going to be protected using critical sections, I suggest we 
investigate a way to minimize the number of critical sections we enter and 
exit for statistics purposes, ideally minimizing and amortizing the entry/exit 
of critical sections.  The more time you spend in critical sections, the more 
likely it is an interrupt will fire during the critical section, and our 
critical sections are optimized to optimistically assume that interrupts won't 
fire during critical sections.

FWIW, One of the reasons some of the lower level network statistics haven't 
been modified to occur in critical sections or locked regions is that there 
are still open design questions about locking for network interfaces and 
address lists.  It's likely that once that work is done, locking of the 
statistics can be done "for free" by piggy-backing it on other necessary work.

I've wanted to see 64-bit statistics on our network interfaces for a long time 
-- I wrap 32-bit counters very quickly, myself, and it will only happen more 
quickly in the future!

Robert N M Watson
Computer Laboratory
University of Cambridge