Re: FreeBSD system profiling and tuning for 10, 11 and 12

On Sat, Dec 17, 2016 at 04:06:40PM +0800, Julian Elischer wrote:

> Hi
> 
> I'm looking for recent information regarding profiling and tuning in 
> FreeBSD.
> 
> Google has turned up some links but I think that the best leads are 
> still hiding..
>   for example I only found 
> https://wiki.freebsd.org/NetworkPerformanceTuning recently.
> 
> (BTW Anyone who has a moment is encouraged to check if they have 
> anything to add to it.)

===
Ensure BPF is OFF. No tcpdump, cdpd, lldpd, dhcpd, dhcp-relay. Patches
are coming. Check netstat -B.
===

Don't see any problem with tcpdump up to 40G.
tcpdump used for capture selected stream.

===
On FreeBSD older than 11.0, use more compact ip_fastfoward routine. It
processes most packets falling back to 'normal' forward routine for
fragments, packets with options, etc. Can save you up to 20% speed
(but will break IPSec).

    net.inet.ip.fastforwarding=1
===

I am not found in 11/12 this sysctl/tunable.

===
Skip feeding /dev/random from network by adding harvest_mask="351" to
/etc/rc.conf or theses line to /etc/sysctl.conf:

    kern.random.sys.harvest.ethernet=0
    kern.random.sys.harvest.point_to_point=0
    kern.random.sys.harvest.interrupt=0
===

On 11:
# sysctl kern.random.sys.harvest
sysctl: unknown oid 'kern.random.sys.harvest'
 
> I am sure there is better information around for profiling the kernel 
> and modules,
> but I am not seeing "the definitive profiler's guide" out there. I do 
> know several people
> have blogs that cover this sort of thing. If you have one or know of good
> profiling resources we should gather them.. send them to me and I'll 
> try make
> sure everything is up to date, and put them together in a wiki page.
> 
> there is already some stuff there,
> (see 
> https://wiki.freebsd.org/NetworkPerformanceTuning?action=fullsearch&context=180&value=profiling&fullsearch=Text 
> )
> but it could do with gathering together.

You might surprised, but various options in BIOS can gain up to 10-20%%
every, 50% total.

Also, be careful about using internal architectures detials.
I am already overloaded:

a) bandwidtch of DMI2 link
b) bandwidtch of 8x PCIe3

Not all cores equals for IRQ handling and cost of PCIe access.