Re: EM and TSO

On 5/17/07, Danny Braniss <danny_at_cs.huji.ac.il> wrote:
> > On 5/16/07, Thomas Hurst <tom.hurst_at_clara.net> wrote:
> > > * Jack Vogel (jfvogel_at_gmail.com) wrote:
> > >
> > > >  I introduced a change yesterday that limited TSO to PCI Express
> > > >  adapters, I did this more for avoidance rather than a bug fix, and
> > > >  I'm not 100% sure its the right thing, so I thought I would poll
> > > >  everyone, do you have a PCI-X adapter and are using TSO without
> > > >  problems and wish to keep the support in?
> > > >
> > > >  If no one is then I'll just leave it as is.
> > >
> > > We put CURRENT on one of our Sun X4200's for some MySQL tests a few days
> > > ago and haven't noticed any problems, though it's only handled a few
> > > million packets so far.
> > >
> > > Does this change remove support or just disable it by default?
> >
> > It removed it, but Andre and my own guilty conscience has convinced
> > me that I should change it back, once I get an MSI/X bug that was
> > discovered with the code solved there will be another delta that will
> > reenable it for PCI-X again.
>
> em0_at_pci1:1:0:   class=0x020000 card=0x10118086 chip=0x10108086 rev=0x03
> hdr=0x00
>     vendor     = 'Intel Corporation'
>     device     = '82546EB Dual Port Gigabit Ethernet Controller (Copper)'
>     class      = network
>     subclass   = ethernet
>     cap 01[dc] = powerspec 2  supports D0 D3  current D0
>     cap 07[e4] = PCI-X 64-bit supports 133MHz, 2048 burst read, 1 split
> transaction
>     cap 05[f0] = MSI supports 1 message, 64 bit
>
> I have been testing TSO on a SunFire X4100, and though it works, I notice
> that setting it on actually reduces throughput, from an average of 55MB/s
> to 24MB/s
>
> any explanation?

TSO is workload sensitive and its not always the right answer.
The whole realm of performance tweaking has been outside my
scope right now, but it needs to be done at some point.

Most the testing I did while developing the code was on PCI-E,
and on say an 82571 I saw no significant reduction of throughput,
with a noticeable improvement in CPU. The adapter you are testing
on is one that I just disabled by default because I dont think it is
capable of showing TSO to its best advantage.

The has been no real work on tweaking this stuff to understand its
best usage yet, it would be a great task for someone to take on :)

Jack