Re: HEADS-UP: IFLIB implementations of sys/dev/e1000 em, lem, igb pending

 ---- On Wed, 11 Jan 2017 12:02:06 -0800 Sean Bruno <sbruno_at_freebsd.org> wrote ---- 
 >  
 >  
 > On 01/11/17 12:47, Olivier Cochard-Labbé wrote: 
 > > On Wed, Jan 11, 2017 at 4:17 PM, Sean Bruno <sbruno_at_freebsd.org 
 > > <mailto:sbruno_at_freebsd.org>> wrote: 
 > >  
 > >  
 > >  
 > >     Olivier: 
 > >  
 > >     Give this a quick try.  This isn't the correct way to do this, but I 
 > >     want to see if I'm on the right path: 
 > >  
 > >  
 > > ​thanks, it fix the problem, I've got back the 4 queues:​ 
 > >  
 > > ​igb2: <Intel(R) PRO/1000 PCI-Express Network Driver> port 0x3000-0x301f 
 > > mem 0xdfea0000-0xdfebffff,0xdff24000-0xdff27fff irq 18 at device 20.0 on 
 > > pci0 
 > > igb2: attach_pre capping queues at 8 
 > > igb2: using 1024 tx descriptors and 1024 rx descriptors 
 > > igb2: msix_init qsets capped at 8 
 > > igb2: pxm cpus: 4 queue msgs: 9 admincnt: 1 
 > > igb2: using 4 rx queues 4 tx queues 
 > > igb2: Using MSIX interrupts with 5 vectors 
 > > igb2: allocated for 4 tx_queues 
 > > igb2: allocated for 4 rx_queues 
 > > igb2: Ethernet address: 00:08:a2:09:33:da 
 > > igb2: netmap queues/slots: TX 4/1024, RX 4/1024 
 > > ​ 
 > > In forwarding mode, I measure about 10% performance drop with this new 
 > > drivers on this hardware: 
 > >  
 > > x head r311848: packets per second 
 > > + head r311849 and BAR patch: packets per second 
 > > +--------------------------------------------------------------------------+ 
 > > |++    ++ +                                                   xxx     x   x| 
 > > |                                                            |__M__A____|  | 
 > > | |___AM__|                                                                | 
 > > +--------------------------------------------------------------------------+ 
 > >     N           Min           Max        Median           Avg        Stddev 
 > > x   5        924170        943071        927509      931612.1     8096.8269 
 > > +   5        831452      845929.5        840940      838730.5     6413.5602 
 > > Difference at 95.0% confidence 
 > >         -92881.6 +/- 10652.2 
 > >         -9.96999% +/- 1.07481% 
 > >         (Student's t, pooled s = 7303.85) 
 > >  
 > > Regards, 
 > >  
 > > Olivier 
 > >  
 >  
 >  
 > Hmmm ... did your old tests do 4 or 8 queues on this hardware? 
 >  
 > Did the old tests run 1024 tx/rx slots or the max 4096? 

That's a great point, only having one thread per core could easily account for this. I'm hoping Sean can make txq != rxq work so that you can have 8txqs and 4 rxqs.

-M