Re: Device polling

On Sun, Jun 13, 2004 at 04:07:13PM -0500, Jon Noack wrote:
> On 06/13/04 13:13, Ruslan Ermilov wrote:
> >On Sun, Jun 13, 2004 at 01:07:28PM -0500, Jon Noack wrote:
> >>I just tested this on my SMP all-in-one home server (Web, Mail,
> >>NFS, Samba, Squid, etc.).  It's been up for over 24 hours with no
> >>apparent issues.  The machine is used pretty heavily, with NFS
> >>mounted home directories and CVS mirror (see below) -- a CVS update
> >>of the src tree over NFS has done a good job of breaking fragile
> >>setups in the past. Everything seemed OK.  That said, peak
> >>performance (as tested by iperf) took a nosedive: with 32-bit em
> >>adapters (gige), tcp bandwidth dropped from over 360Mbps to around
> >>200 Mbps.  If anyone has any suggestions for more in-depth testing,
> >>I'd be willing to try them.  If I have the time I may also try the
> >>latest netperf patch and see how that affects things.
> >
> >What are your operational polling(4) parameters?  What the HZ is set
> >to?
> 
> HZ=1000 and I was using the defaults for polling.  Bumping up burst_max 
> to 300 results in slightly better peak performance at 220 Mbps. 
> polling(4) says burst_max=150 is good for HZ=1000 and 100 Mbit networks; 
> are there any recommendations (for burst_max or in general) for 1 Gbit 
> networks?
> 
> $ sysctl kern.polling
> kern.polling.burst: 150
> kern.polling.each_burst: 5
> kern.polling.burst_max: 150
> kern.polling.idle_poll: 0
> kern.polling.poll_in_trap: 0
> kern.polling.user_frac: 50
> kern.polling.reg_frac: 20
> kern.polling.short_ticks: 2000
> kern.polling.lost_polls: 403164
> kern.polling.pending_polls: 0
> kern.polling.residual_burst: 0
> kern.polling.handlers: 1
> kern.polling.enable: 1
> kern.polling.phase: 0
> kern.polling.suspect: 365727
> kern.polling.stalled: 0
> kern.polling.idlepoll_sleeping: 1
> 
Read the polling(4) manpage very carefully, to understand
what these various parameters mean.  Experiment.

Bump HZ.  If you were using the 64-bit PCI I'd say bump it
to 5000, but since you're using 32-bit PCI, bumping it that
high just doesn't make sense, so raise it up to 2000-3000.

Set kern.polling.idle_poll=1.

Set kern.polling.user_frac=30.

Raising up kern.polling.burst_max might help as well.

Remember, the maximum number of packets each interface can
receive per second with polling enabled is (burst_max * HZ).

When tuned properly, and if the card is not rl(4), you will
usually get better peak performance in polling mode than in
interrupt mode.


Cheers,
-- 
Ruslan Ermilov
ru_at_FreeBSD.org
FreeBSD committer