question about libc fopen

From: Ê¢»Û»ª <hhsheng_at_corp.netease.com>
Date: Fri, 23 Dec 2016 17:23:02 +0800
hi all,

   hi~
   we are from Chinese Game Develop Corp, Netease.
   and One of our product using FreeBsd as its OS platform.
   This Game has Millions of players online , and Each Server may holds 25000+ tcp connection at the same time.Thanks to BSD and kqueue :)

   it's one of our server , netstat cmd to list connections overall...
   netstat -an | grep 13396 (it's our listening port) | wc -l 
   23221

    recently we do some performance optimize and promote this connection limit to 28000+ or 30000+.
  But we find Freebsd has a limit that this huge online number will take 28000+ fd, and bsd libc fopen FILE * struct's 
  fileno only support to SHORT . such as ..

struct __sFILE {
 ...
 short _file; /* (*) fileno, if Unix descriptor, else -1 */
 ...

  so if our server want to fopen some file when we still hold this online number, the fopen's fileno may easily exceed
32767, and fopen definitely return a err code. then the server will appear some fataly ERROR. 

  for example, if my server tcp already holds 32000 connection
  fopen only has 767 fd to use

  the problem has no bussiness with tcp fd, BUT fopen ...

  in some particular situlations , my server will open 1k+ FILE , that will exceed the fileno limit, and overflow occur
  my server can't open any file more ,that's the problem ~

  we do a simple test and confirm this situation.

  then in fopen's code , we notice that we can use open to return a fd instread of using fopen to avoid this overflow, 
   as below

68 /* 
1 * File descriptors are a full int, but _file is only a short. 
2 * If we get a valid file descriptor that is greater than 
3 * SHRT_MAX, then the fd will get sign-extended into an 
4 * invalid file descriptor. Handle this case by failing the 
5 * open. 
6 */

  BUT ... so many libc FILE series function needs a FILE * pointer as input argument, we can't convert all of them to fd, 
or it will be a rather suffering things to us. 

  and even in BSD 10 , it seems this SHORT fileno limit still there , but other OS as debian , FILE strucnt's fileno is a int .

  so i felt if bsd official could change libc FILE struct's fileno to a UNSIGNED SHORT that may be an effecient and convenient solution just for my case ?
  UNSIGNED SHORT fileno is enough for me~~~ or i should modify it by myself ?

  LoL, thank you !!!!!

yours sincerely
                                       winson sheng


winson sheng
Received on Fri Dec 23 2016 - 09:13:52 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:09 UTC