Re: Knobs to reduce PCI interrupt latency

From: M. Warner Losh <imp_at_bsdimp.com>
Date: Wed, 04 Mar 2009 01:50:13 -0700 (MST)
In message: <200903040942.39191.hselasky_at_c2i.net>
            Hans Petter Selasky <hselasky_at_c2i.net> writes:
: Do we have any knobs in FreeBSD to reduce the interrupt latency? I am 
: currently seeing performance differences between 8.x and 7.x. Anyone have any 
: ideas?

If we did, don't you think they would be enabled by default :)

Seriously, I measured interrupt latency on a 7.0-current system at
around 20us for a fast interrupt/filter (300MHz or 400MHz low-power
CPU).  For fast machines, this can approach 1us, which is hard to
measure with the setup I had for the 300MHz case.

Deferring work to the scheduled interrupt can be useful, especailly if
you can keep the pipeline full such that the filter can kick off the
next set of transactions quickly, and then call the completion
routines on the last set in parallel in the ithread.  This works well
for most random-access things, but less well for single threaded,
sequential benchmarks.

Warner


: --HPS
: 
: On Wednesday 04 March 2009, Artyom Mirgorodsky wrote:
: > >Repeat the same test using FreeBSD -current.
: > >
: > >a) On the machine where it is slow.
: >
: > vmstat -i ; sleep 1 ; vmstat -i
: > interrupt                          total       rate
: > irq1: atkbd0                         233          2
: > irq14: ata0                           85          0
: > irq16: vgapci0                      5377         52
: > irq21: hdac0 ohci0                   742          7
: > irq22: nfe0 ehci0                  23610        229
: > irq23: atapci1                      5405         52
: > cpu0: timer                       203959       1980
: > cpu1: timer                       200914       1950
: > Total                             440325       4275
: > interrupt                          total       rate
: > irq1: atkbd0                         234          2
: > irq14: ata0                           85          0
: > irq16: vgapci0                      5439         52
: > irq21: hdac0 ohci0                   742          7
: > irq22: nfe0 ehci0                  24621        236
: > irq23: atapci1                      5405         51
: > cpu0: timer                       205981       1980
: > cpu1: timer                       202937       1951
: > Total                             445444       4283
: >
: > >I think the reduced performance can be explained by a clamp on the
: > > interrupt rate around 1000 interrupts per second instead of 8000. Maybe
: > > someone has an explanation for this?
: >
: > You right, the interrupt rate around 1000 (1011) on this machine, but on
: > FreeBSD 7.1 more 3000. If it is some kind of interrupt aggregation, may be
: > I can try to turn it off?
: >
: > >b) On the machine where it is fast.
: >
: > vmstat -i ; sleep 1 ; vmstat -i
: > interrupt                          total       rate
: > irq4: uart0                         4154          0
: > irq14: ata0                       472922          0
: > irq15: ata1                           26          0
: > irq18: em0                        752711          0
: > irq21: ahc0                           53          0
: > irq23: ehci0                      103456          0
: > irq24: em1                           147          0
: > cpu0: timer                   1551216517       2000
: > cpu1: timer                   1551195251       2000
: > Total                         3103745237       4001
: > interrupt                          total       rate
: > irq4: uart0                         4154          0
: > irq14: ata0                       472923          0
: > irq15: ata1                           26          0
: > irq18: em0                        752713          0
: > irq21: ahc0                           53          0
: > irq23: ehci0                      110949          0
: > irq24: em1                           147          0
: > cpu0: timer                   1551218551       2000
: > cpu1: timer                   1551197285       2000
: > Total                         3103756801       4001
: 
: 
: _______________________________________________
: freebsd-usb_at_freebsd.org mailing list
: http://lists.freebsd.org/mailman/listinfo/freebsd-usb
: To unsubscribe, send any mail to "freebsd-usb-unsubscribe_at_freebsd.org"
: 
: 
Received on Wed Mar 04 2009 - 07:52:34 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:39:43 UTC