RE: Why is em nic generating interrupts?

Michael O. Boev wrote:
> From: Terry Lambert [mailto:tlambert2_at_mindspring.com]
> > "Michael O. Boev" wrote:
> > > I've got a [uniprocessor 5.1-RELEASE] router machine with fxp
> > > and em nics.
> > > I've built my kernel with the following included:
> > >
> > > options         DEVICE_POLLING
> > > options         HZ=2500
> > >
> > > and enabled polling in /etc/sysctl.conf.
> > [ ... ]
> > > What's happening? Is polling working in my case?
> > > If yes, why is vmstat showing interrupts? I see clearly,
> > > that fxp's counter doesn't increase, and em's is constantly growing.
> > >
> > > Is there anyone who knows for sure that em's polling works?
> >
> > You may want to ask Luigi; polling is his code.
> >
> > However, I believe the issue is that polling doesn't start
> > until you take an interrupt, and it stops as soon as there is
> > no more data to process, and waits for the next interrupt.
> >
> > If you were to jack your load way up, you would probably see
> > an increase in interrupts, then them dropping off dramatically.
> To this dare I object, that there is traffic going through this machine,
> and fxp0 is NOT generating interrupts, while em IS. So, if the rule above
> works, they both have to behave in same ways.

This is the third time I've received this message.  I guess you feel some
urgency about receiving a reply?

My answer is this:  I did the polling code for a number of the drivers, and
though it was committed by someone else, I can guarantee you that it
will have interrupts, unless the code was implemented incorreectly, since
the code is meant to  do this polling only under heavy load.

The purpose of going to a polling model is *not* to avoid interrupts, it's
to avoid processing data into the input routines when you don't have
enough CPU cycles to deal with it before more data comes in.  In other
words, it's to deal with the situation where you would end up in receiver
livelock as a result of taking so many interrupts that you never get an
oportunity to process the packets out of the input queue into the per
protocol input routines at NETISR, or from the top-of stack (socket
buffers) into the user buffers in user space, or run the code in user space
to take the data and generate a reply, or in kernel space to send the
reply out.

In other words, you *only ever* want to do polling when you have sufficient
load to make polling worthwhile, and when you do it under low loads, you
are actually *wasting* cycles, rather than *saving* them.

Polling never *saves* cycles, per se, it only causes incoming data to be
dropped (and, presumably, retransmitted) by the other end of the
connection.

For this, because there is not specific card support for it, and there's no
interface entry points to deal with it, even if there were, the DMA engine on
the card is *NOT* halted, you *still* eat bus transfer overhead, and it still
eats up your PCI bandwidth, and this cost is *incredibly* larger than just
interrupt and requeueing overhead.

If your FXP is not generating any interrupts at all, i think that the polling
code in it is probably broken.

Note: I believe this might be one of the ones I modified, since Luigi's code
originally only dealt with Gigabit NICs, though I may be mistaken.  If so,
my statement that it's broken should be considered to be pretty
authoritative.  8-).

-- Terry