Re: TCP server app performance

On 08/14/2018 09:30 PM, Navdeep Parhar wrote:
> On 8/12/18 9:50 AM, Honda Michio wrote:
>> Hi,
>>
>> I'm measuring TCP server app performance using my toy web server.
>> It just accept TCP connections and responds back HTTP OK to the clients.
>> It monitors sockets using kqueue, and processes each ready descriptor using
>> a pair of read() and write(). (in more detail, it's
>> https://github.com/micchie/netmap/tree/paste/apps/phttpd)
>>
>> Using 100 persistent TCP connections (the client sends 44 B HTTP GET and
>> the server responds with 151 B of HTTP OK) and a single CPU core, I only
>> get 152K requests per second, which is 2.5x slower than Linux that runs the
>> same app  (except that it uses epoll instead of kqueue).
>> I cannot justify this by myself. Does anybody has some intuition about how
>> much FreeBSD would get with such workloads?
>> I tried disabling TCP delayed ack and changing interrupt rates, but no
>> significant difference was observed.
>>
>> I use FreeBSD-CURRENT with GENERIC-NODEBUG (git commit hash: 3015145c3aa4b).
>> For hardware, the server has Xeon Silver 4110 and Intel X540 NIC (activate
>> only a single queue as I test with a single CPU core). All the offloadings
>> are disabled.
> 
> I hope hw L3/L4 checksumming is still on?
They are off but exchanged messages are too small to benefit from these 
offloads anyways.
> 
> Are your results similar to what you get with 100 (same number as your
> test clients) netperf's doing TCP_RR on this setup, or wildly different?
I cannot find any netperf option that sends multiple connections with 
TCP_RR. I also don't find any equivalent in iperf. But single-connection 
performance is similar to what I get with wrk and my HTTP server.

Running 100 netperf processes does not show aggregate throughput, and 
that also means running 100 kevent() loops, which is quite different 
behaviour from what I want.

Cheers,
- Michio

> 
> Regards,
> Navdeep
>