RE: serious networking (em) performance (ggate and NFS) problem

From: Daniel Eriksson <daniel_k_eriksson_at_telia.com>
Date: Thu, 18 Nov 2004 18:49:08 +0100
M. Warner Losh wrote:

> Also, make sure that you aren't sharing interrupts between
> GIANT-LOCKED and non-giant-locked cards.  This might be exposing bugs
> in the network layer that debug.mpsafenet=0 might correct.  Just
> noticed that our setup here has that setup, so I'll be looking into
> that area of things.

I have a Tyan Tiger MPX board (dual AthlonMP) that has two 64bit PCI slots.
I have an Adaptec 29160 and a dual port Intel Pro/1000 MT plugged into those
slots.

As can be seen from the vmstat -i output below, em1 shares ithread with
ahc0. This is with ACPI disabled. With ACPI enabled all devices get their
own ithread (I think, not 100% sure). However, because of some hardware
problem (interrupt routing?), em0 interrupts will somehow leak into
atapci1+, generating a higher interrupt load. I'm not sure how expensive
this is.

Finally, my question. What would you recommend:
1) Run with ACPI disabled and debug.mpsafenet=1 and hope that the mix of
giant-safe and giant-locked (em and ahc) doesn't trigger any bugs. This is
what I currently do.
2) Run with ACPI disabled and debug.mpsafenet=0 and accept lower network
performance (it is a high-traffic server, so I'm not sure this is a valid
option).
3) Run with ACPI enabled and debug.mpsafenet=1 and accept that em0
interrupts "leak" to the atapci1+ ithread. This I have done in the past.


# vmstat -i
interrupt                          total       rate
irq1: atkbd0                           2          0
irq4: sio0                           710          0
irq6: fdc0                             8          0
irq8: rtc                       11470937        127
irq13: npx0                            1          0
irq14: ata0                      1744545         19
irq15: ata1                      1749617         19
irq16: em0 atapci1+            186062858       2074
irq17: em1 ahc0                 27028088        301
irq18: atapci3                   7393468         82
irq19: atapci4+                  7129446         79
irq0: clk                      179054582       1995
Total                          421634262       4699

/Daniel Eriksson
Received on Thu Nov 18 2004 - 16:49:15 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:38:22 UTC