[SOLVED] Re: net/asterisk13: memory leak under 12-CURRENT?

From: O. Hartmann <ohartmann_at_walstatt.org>
Date: Fri, 29 Sep 2017 11:18:37 +0200
On Thu, 28 Sep 2017 07:57:01 +0200
"O. Hartmann" <ohartmann_at_walstatt.org> wrote:

> On Wed, 27 Sep 2017 12:51:18 +0200
> Hans Petter Selasky <hps_at_selasky.org> wrote:
> 
> > On 09/27/17 09:05, Guido Falsi wrote:  
> > > On 09/26/2017 15:41, O. Hartmann wrote:    
> > >> On Tue, 26 Sep 2017 15:06:23 +0200
> > >> Guido Falsi <madpilot_at_FreeBSD.org> wrote:    
> > >     
> > >> Since I run net/asterisk with automatic module loading (I'm new to
> > >> asterisk), this is very likely and might cause the problem somehow.
> > >>    
> > > 
> > > You can exclude single modules from autoloading via modules.conf.
> > >     
> > >>> Not sure, restarting the daemon should free any leaked memory the daemon
> > >>> has. If a killed process leaves memory locked at the system level there
> > >>> should be some other cause.    
> > >>
> > >> Even with no runnidng asterisk, memory level drops after the last
> > >> shutdown of asterisk and keeps that low. Even for weeks! My router never
> > >> shows that high memory consumption, even under load.    
> > > 
> > > But while asterisk is running does the memory usage increase unbounded
> > > till filling all available memory or does it stabilize at some point?
> > > 
> > > Asterisk is relatively memory hungry, especially with all modules
> > > enabled. It also caches and logs various information in RAM, even doing
> > > "nothing" it will cache and log that "nothing" activity. If memory does
> > > stabilize after some point it's not really a leak but it's standard
> > > memory usage. To reduce it you should disable all unused modules.
> > >     
> > >>
> > >> The question would be: how to use vmstat to give hints for those familiar
> > >> with memory subsystems to indicate a real bug?
> > >>
> > >> I tried to find some advices, but maybe my English isn't good enough to
> > >> make google help.    
> > > 
> > > I'm not able to give you a correct indication, but if the memory usage
> > > is not increasing indefinitely but is stabilizing I'd say it's not
> > > really a leak.
> > >     
> > 
> > Did you look at the output from "vmstat -m" and "vmstat -z" ?
> > 
> > --HPS  
> 
> I did not, but now I will ;-)
> 
> Thanks for the hint!
> 
> Kind regards,
> Oliver

For the completeness.

Guido already explained that things I considered "stange' could be normal. I
think I missed the point, that in some cases, caching data can also persist
even when the process, which I considered to be the consumer, died.

The vmstat output is taken after a couple of restarts of asterisk and while
asterisk was still running.

I had a very simplistic, outdated memory model in mind and Guido made that
clear that it isn't.

I was concerned about if it comes to the use of a small SoC with only 1 or 2 GB
of RAM.

Sorry for the noise :-(

Kind regards,

Oliver

[...]

last pid: 19557;  load averages:  0.22,  0.18,  0.14                                                                                                       up 1+14:17:56  11:14:44
25 processes:  1 running, 24 sleeping
CPU:  0.3% user,  0.0% nice,  0.8% system,  0.0% interrupt, 98.9% idle
Mem: 27M Active, 104M Inact, 392M Wired, 146M Buf, 3421M Free
Swap: 8192M Total, 8192M Free

  PID USERNAME       THR PRI NICE   SIZE    RES STATE   C   TIME    WCPU COMMAND
36617 asterisk        70  20    0   147M 74236K piperd  2   3:38   2.42% asterisk
19557 root             1  20    0 13180K  3508K CPU3    3   0:00   0.13% top
93400 root             1  20    0 11444K  2100K select  3   1:34   0.06% syslogd
12569 root             1  20    0 15136K  3576K select  1   4:35   0.04% ppp
88182 administrator    1  20    0 18768K  7980K select  0   0:00   0.02% sshd
24901 root             1  20    0 10840K  1704K select  1   0:19   0.01% powerd
17169 root             1  20    0 18192K 18296K select  2   0:10   0.01% ntpd
10719 root             1  20    0 14724K  4712K bpf     1   0:03   0.00% arpwatch
 9691 bind             7  52    0 64552K 38192K sigwai  3   0:19   0.00% named
98424 root             1  20    0 32100K 20948K nanslp  2   0:05   0.00% perl
39431 asterisk         1  52    0 13848K  5072K select  0   0:01   0.00% mpg123
73938 root             1  20    0 11332K  2036K nanslp  3   0:01   0.00% cron
45343 daemon           1  20    0 11316K  1940K select  2   0:00   0.00% rpcbind
41607 root             1  20    0 10976K  1848K kqread  1   0:00   0.00% autounmountd
82722 root             1  22    0 18396K  7920K select  1   0:00   0.00% sshd
97178 root             1  21    0 16216K  6780K select  2   0:00   0.00% sudo
97625 root             1  20    0 13604K  4356K pause   2   0:00   0.00% csh
46309 root             1  20    0 10880K  1784K autofs  3   0:00   0.00% automountd
90914 administrator    1  20    0 13172K  4184K pause   0   0:00   0.00% csh
40858 asterisk         1  52    0 13616K  3040K pipewr  3   0:00   0.00% mpg123
97392 root             1  23    0 11776K  3096K wait    3   0:00   0.00% su
77979 dhcpd            1  20    0 21896K  8340K select  0   0:00   0.00% dhcpd
69003 root             1  20    0 18000K  4904K select  0   0:00   0.00% sshd
52102 root             1  52    0 10880K  1788K ttyin   2   0:00   0.00% getty
97105 root             1  20    0 10868K  2444K ttyin   3   0:00   0.00% getty

[vmstat -m]
root_at_gate:~ # vmstat -m
         Type InUse MemUse HighUse Requests  Size(s)
     acpiintr     1     1K       -        1  64
       acpica  2689   269K       -    53492  16,32,64,128,256,512,1024
        vtbuf     8   656K       -       14  4096
           vt     3     2K       -        3  512
       autofs     6     9K       -  8219485  16,128,8192
     acpitask     1    64K       -        1  65536
      acpisem    24     3K       -       24  128
      acpidev    32     2K       -       32  64
      CAM SIM     3     1K       -        3  256
      CAM XPT    18     2K       -      112  32,64,128,256,512,1024,2048,65536
      CAM DEV     4     8K       -       10  2048
       DEVFS3   140    35K       -      520  256
       DEVFS1   109    55K       -      349  512
   DEVFS_RULE    55    26K       -       55  64,512
        DEVFS    11     1K       -       12  16,128
       DEVFSP     7     1K       -        7  64
NFSD V4client     1     1K       -        1  256
 NFSD lckfile     1     1K       -        1  256
 NFSD session     1     1K       -        1  1024
    pfs_nodes    20     5K       -       20  256
  tmpfs mount     2     1K       -        2  128
   tmpfs name  1500    30K       -     2098  16,32,64
     eli data     7     2K       -       29  64,256,512,1024
      CAM CCB     0     0K       -   117024  2048
geom_flashmap     0     0K       -       24  256
         GEOM   278    42K       -    12066
16,32,64,128,256,512,1024,2048,4096,8192,16384 raid_data     0     0K
-     2088  32,128,256 isadev     6     1K       -        6  128
     CAM path     5     1K       -       38  32
   CAM periph     4     1K       -       19  16,32,64,128,256
    acpi_perf     4     1K       -        4  128
         cdev     4     1K       -        4  256
     filedesc     7    85K       -      276  128,4096,65536
        sigio     0     0K       -        1  64
     filecaps     0     0K       -    25082  16,32,64
         kenv   119    12K       -      122  16,32,64,128,8192
       kqueue    61    17K       -    16840  64,256,512,2048,8192
    proc-args    56     3K       -    29790  16,32,64,128,256
        hhook    13     3K       -       13  32,256
      ithread   130    26K       -      130  32,128,256
       linker   127     9K       -      129  16,32,64,512
        lockf    23     3K       -     5019  64,128
   loginclass     3     1K       -        3  64
        cache     1     1K       -        1  32
       devbuf 16929 33991K       -    17350
16,32,64,128,256,512,1024,2048,4096,8192 temp    75     5K       -    79184
16,32,64,128,256,512,1024,2048,4096,8192,16384,65536 CAM I/O Scheduler
1     1K       -        1  1024 module   327    41K       -      327  128
     mtx_pool     2    16K       -        2  8192
          osd     3     1K       -        9  16,32,64,128,256
     pmchooks     1     1K       -        1  128
         pgrp    23     3K       -      922  128
      session    19     3K       -      359  128
         proc     2    32K       -        2  16384
      subproc   162   267K       -    16726  512,4096
         cred   111    28K       -    16500  256
       kbdmux     5    22K       -        5  512,1024,2048,16384
       plimit    22     6K       -     5489  256
      uidinfo     9     5K       -      287  128,4096
       sysctl     0     0K       -    21212  16,32,64
    sysctloid  5895   303K       -     5975  16,32,64,128
    sysctltmp     0     0K       -     1183  16,32,64,256,1024
      tidhash     1    32K       -        1  32768
      callout     5  2184K       -        5  
         umtx   560    70K       -      560  128
     p1003.1b     1     1K       -        1  16
         SWAP     2   549K       -        2  32
          bus  2646   191K       -     5020  16,32,64,128,256,1024
       bus-sc    61  1729K       -     1214
16,32,64,128,256,512,1024,2048,4096,8192,16384,65536 devstat    16    33K
-       16  32,4096 eventhandler   107     9K       -      107  64,128
   gtaskqueue    30    27K       -       30  16,32,256,8192
         kobj   131   524K       -     3199  4096
      Per-cpu     1     1K       -        1  32
         rman   164    19K       -      553  32,128
         sbuf     1     1K       -     9894
16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536 toponodes    14
2K       -       14  128 taskqueue    36     4K       -       48  16,32,256
     terminal     3     1K       -        3  256
       Unitno    41     3K       -     7477  32,64
         vmem     3   200K       -        5  4096,8192,65536
     ioctlops     1     8K       -      297  256,1024,8192
       select   181    23K       -      280  128,8192
          iov     0     0K       -  2135314  16,64,128,256
          msg     4    30K       -        4  2048,4096,8192,16384
          sem     4   106K       -        4  2048,4096
          shm     1    32K       -        1  32768
          tty     7     7K       -       10  1024
          pts     1     1K       -        4  256
     mbuf_tag   175    11K       - 84049978  32,64,256
        shmfd     1     8K       -        1  8192
       soname     4     1K       -  2413713  16,32,128
          pcb    37   663K       -    33716  16,32,64,128,1024,2048,8192
          acl     0     0K       -     3696  4096
     vfscache     4  2097K       -        4  256,16384,32768
   cl_savebuf     0     0K       -      956  64,128,256
     vfs_hash     1  1024K       -        1  
       vnodes     1     1K       -        1  256
        mount    98     5K       -      539  16,32,64,128,256
       statfs     0     0K       -    32609  4096
  vnodemarker     0     0K       -    33467  512
      GSS-API     1     1K       -        1  64
chacha20random     1     1K       -        1  1024
        iconv     2     1K       -        2  128
          BPF    42  1079K       -       43  16,64,128,512,4096
      BPF_JIT     7     1K       -       15  16,32,64
        ifnet    15    29K       -       15  128,2048
       ifaddr    69    13K       -       70  16,32,256,512,4096
  ether_multi    36     2K       -       36  16,64
        clone    17     3K       -       17  128
        ipsec     1     1K       -        1  256
      lltable    38    12K       -      624  256,512
         sppp     1     2K       -        1  2048
          tun     1     1K       -        1  256
         vlan    14     2K       -       14  64,128
      entropy     0     0K       -    15571  4096
        iflib   132   816K       -      144
32,64,128,1024,2048,8192,16384,32768 routetbl    96    19K       -      123
32,128,256,512,4096 vnet     1     1K       -        1  64
    vnet_data     1    64K       -        1  65536
vnet_data_free     1     1K       -        1  32
     netgraph     9     1K       -        9  64
 netgraph_msg     0     0K       -       22  64,128,256,512,1024
netgraph_hook     4     1K       -        6  128
netgraph_node    14     4K       -       17  128,256
netgraph_pppoe     2    17K       -        5  64,512,16384
netgraph_path     0     0K       -        9  16
netgraph_sock     3     1K       -        6  32,128
         igmp    14     2K       -       14  128
         ipid     2    24K       -        2  8192,16384
   in_mfilter     1     1K       -        1  1024
     in_multi     8     2K       -        8  256
  ip_moptions     2     1K       -        2  64,256
encap_export_host     2     2K       -        2  1024
    mroutetbl     1     1K       -        1  256
    sctp_a_it     0     0K       -        8  16
     sctp_vrf     1     1K       -        1  64
     sctp_ifa     8     1K       -        9  128
     sctp_ifn     8     1K       -        8  128
    sctp_iter     0     0K       -        8  256
    hostcache     1    32K       -        1  32768
          LRO    24   480K       -       24  8192,32768
      tcpfunc     1     1K       -        1  64
     syncache     1    68K       -        1  
     libalias    66   136K       -    31040  128,65536
      sctpnat     6    80K       -        6  8192,16384
  inpcbpolicy    32     1K       -    37745  32
     secasvar     1     1K       -        1  1024
       sahead     1     1K       -        1  1024
  ipsecpolicy     2     2K       -        2  256,1024
    ipsec-saq     2     2K       -        2  1024
     dummynet     4     4K       -        4  512,1024
     dummynet    13     6K       -   278817  256,512
         UART     6     5K       -        6  16,1024
  IpFw/IpAcct    96    59K       -      163
16,32,64,128,256,512,1024,2048,4096,8192,16384 ipfw_tbl    10     2K
-       10  128 crypto     1     1K       -        1  512
        xform     0     0K       -      986  16,32,256
          rpc     2     8K       -        2  4096
audit_evclass   230     8K       -      286  32
      mactemp     0     0K       -      194  8192
      pagedep     2   256K       -      416  256
     inodedep     2  2048K       -     4035  512
    bmsafemap     2    16K       -     3283  256,8192
       newblk     2  4096K       -     5152  256
     indirdep     0     0K       -     2368  128,32768
     freefrag     0     0K       -      183  128
     freeblks     0     0K       -      367  256
     freefile     0     0K       -      368  64
       diradd     0     0K       -      710  128
        mkdir     0     0K       -        6  128
       dirrem     0     0K       -      706  128
    newdirblk     0     0K       -        3  64
     freework     2     1K       -      772  16,128
      freedep     0     0K       -        4  64
      jaddref     0     0K       -      693  128
      jremref     0     0K       -      684  128
       jmvref     0     0K       -       42  128
      jnewblk     0     0K       -     5097  128
    jfreefrag     0     0K       -      181  128
         jseg     0     0K       -     4829  128
      jsegdep     0     0K       -     6655  64
        sbdep     0     0K       -     1182  64
     savedino     0     0K       -     1317  256
      jblocks     4     1K       -       12  128,256
      softdep     2     1K       -       27  512
  ufs_dirhash   213    44K       -      243  16,32,64,128,256,512,1024
    ufs_quota     1  1024K       -        1  
    ufs_mount     9    30K       -       87  512,2048,4096,8192
    vm_pgdata     1     1K       -        1  128
      UMAHash     7    38K       -       18  512,1024,2048,4096,8192,16384,32768
    CAM queue     7     3K       -       26  16,32,512
          USB    20    23K       -       22  16,128,512,4096
  fpukern_ctx     4     4K       -        4  1024
      memdesc     1     4K       -        1  4096
   aesni_data     7     6K       -        7  32,256,1024
     pci_link    12     1K       -       12  32,128
       cpuctl     1     1K       -        1  32
       USBdev    15     2K       -       15  32,64,128,256
       apmdev     1     1K       -        1  128
   madt_table     0     0K       -        2  32,4096
CAM dev queue     3     1K       -        3  64
      io_apic     2     6K       -        2  2048,4096
   local_apic     1     4K       -        1  4096
          MCA    11     2K       -       11  32,128
         cpus     2     1K       -        2  16
          msi    16     2K       -       16  128
     nexusdev     6     1K       -        6  16
root_at_gate:~ # 








[vmstat -z]
root_at_gate:~ # vmstat -z
ITEM                   SIZE  LIMIT     USED     FREE      REQ FAIL SLEEP

UMA Kegs:               256,      0,     221,       4,     221,   0,   0
UMA Zones:              704,      0,     222,       3,     222,   0,   0
UMA Slabs:               80,      0,    5287,      13,    5512,   0,   0
UMA Hash:               256,      0,       5,      10,      12,   0,   0
4 Bucket:                32,      0,     221,     904,    2102,   0,   0
6 Bucket:                48,      0,      23,    1056,     570,   0,   0
8 Bucket:                64,      0,      42,    1260,   37128,  11,   0
12 Bucket:               96,      0,      74,     705,  469791,   0,   0
16 Bucket:              128,      0,     126,    1207,   70315,   0,   0
32 Bucket:              256,      0,     198,     672,   27016, 135,   0
64 Bucket:              512,      0,     207,     425,    4310, 167,   0
128 Bucket:            1024,      0,     178,     318,  160115,   0,   0
256 Bucket:            2048,      0,     278,      38,   94068,   0,   0
vmem btag:               56,      0,   15673,     586,   17122, 115,   0
VM OBJECT:              240,      0,    7368,     696,  194473,   0,   0
RADIX NODE:             144,      0,    9059,    1147,  409047,   0,   0
MAP:                    232,      0,       3,      65,       3,   0,   0
KMAP ENTRY:             120,      0,       5,     259,       5,   0,   0
MAP ENTRY:              120,      0,    2904,    2607,  808977,   0,   0
VMSPACE:               2496,      0,      26,      28,   16590,   0,   0
fakepg:                 104,      0,       0,       0,       0,   0,   0
mt_zone:              16400,      0,     397,       0,     397,   0,   0
16:                      16,      0,      23,    1232,   36289,   0,   0
16:                      16,      0,    1211,     797,   25573,   0,   0
16:                      16,      0,      25,    1230,      34,   0,   0
16:                      16,      0,     222,    1033, 8220542,   0,   0
16:                      16,      0,    1905,     856,   20512,   0,   0
16:                      16,      0,     122,    1886, 2411743,   0,   0
16:                      16,      0,      92,    1163,   70381,   0,   0
16:                      16,      0,       0,    1255,     385,   0,   0
32:                      32,      0,     105,    1270,   38699,   0,   0
32:                      32,      0,     460,     665,    4210,   0,   0
32:                      32,      0,      59,    1316,   33644,   0,   0
32:                      32,      0,    1107,     768,   27259,   0,   0
32:                      32,      0,    2444,     681,    5616,   0,   0
32:                      32,      0,      66,    1059,     719,   0,   0
32:                      32,      0,     182,     943,    1510,   0,   0
32:                      32,      0,      11,    1114,    7866,   0,   0
64:                      64,      0,      84,     970,      86,   0,   0
64:                      64,      0,     919,     879,   19134,   0,   0
64:                      64,      0,     207,     847,    1124,   0,   0
64:                      64,      0,     688,     614,    1734,   0,   0
64:                      64,      0,     351,    1199,    1545,   0,   0
64:                      64,      0,     136,    1414,   24244,   0,   0
64:                      64,      0,    8325,     789,  362569,   0,   0
64:                      64,      0,     211,    1339,20905924,   0,   0
128:                    128,      0,     662,     857,   23856,   0,   0
128:                    128,      0,    1690,     573,   33432,   0,   0
128:                    128,      0,      37,     490,      85,   0,   0
128:                    128,      0,     835,    1304,   37383,   0,   0
128:                    128,      0,    1749,     762,    3792,   0,   0
128:                    128,      0,     392,     755,   10647,   0,   0
128:                    128,      0,     343,     432,    6647,   0,   0
128:                    128,      0,      33,     742,    5036,   0,   0
256:                    256,      0,      82,     173,    1413,   0,   0
256:                    256,      0,      19,     356,    2226,   0,   0
256:                    256,      0,      28,     227,      30,   0,   0
256:                    256,      0,     250,     365,    6275,   0,   0
256:                    256,      0,     181,     314,   24437,   0,   0
256:                    256,      0,     137,     358,    5862,   0,   0
256:                    256,      0,     106,     329, 1713374,   0,   0
256:                    256,      0,      32,     403,63057061,   0,   0
512:                    512,      0,       1,     319,    4017,   0,   0
512:                    512,      0,     114,     118,     369,   0,   0
512:                    512,      0,     121,      55,     122,   0,   0
512:                    512,      0,      76,      76,     143,   0,   0
512:                    512,      0,      27,     613,  279136,   0,   0
512:                    512,      0,      56,      96,    3584,   0,   0
512:                    512,      0,      41,     111,     100,   0,   0
512:                    512,      0,       4,     148,   33451,   0,   0
1024:                  1024,      0,       1,      51,   16875,   0,   0
1024:                  1024,      0,      18,      34,     909,   0,   0
1024:                  1024,      0,       5,      11,       5,   0,   0
1024:                  1024,      0,      13,     143,     600,   0,   0
1024:                  1024,      0,      48,      44,     152,   0,   0
1024:                  1024,      0,      13,      55,     922,   0,   0
1024:                  1024,      0,      19,      49,      28,   0,   0
1024:                  1024,      0,       0,      28,       2,   0,   0
2048:                  2048,      0,       4,      22,    1622,   0,   0
2048:                  2048,      0,       1,       9,       2,   0,   0
2048:                  2048,      0,       1,       5,       1,   0,   0
2048:                  2048,      0,       2,       4,      25,   0,   0
2048:                  2048,      0,       3,       3,       3,   0,   0
2048:                  2048,      0,       7,      33,  117156,   0,   0
2048:                  2048,      0,      31,      15,      37,   0,   0
2048:                  2048,      0,       4,      22,      48,   0,   0
4096:                  4096,      0,       1,       2,       9,   0,   0
4096:                  4096,      0,       0,       1,       1,   0,   0
4096:                  4096,      0,     194,      16,   19831,   0,   0
4096:                  4096,      0,       9,       8,   32834,   0,   0
4096:                  4096,      0,       2,       7,   19229,   0,   0
4096:                  4096,      0,       3,       3,      18,   0,   0
4096:                  4096,      0,      28,       3,      34,   0,   0
4096:                  4096,      0,       4,       1,       8,   0,   0
8192:                  8192,      0,       0,       1,       1,   0,   0
8192:                  8192,      0,       0,       1,       1,   0,   0
8192:                  8192,      0,       2,       0,       2,   0,   0
8192:                  8192,      0,       4,       8,     340,   0,   0
8192:                  8192,      0,      41,       5,     140,   0,   0
8192:                  8192,      0,       8,       6,     788,   0,   0
8192:                  8192,      0,       3,       3,       9,   0,   0
8192:                  8192,      0,      15,       1,      19,   0,   0
16384:                16384,      0,       2,       2,      11,   0,   0
16384:                16384,      0,       0,       1,       1,   0,   0
16384:                16384,      0,       0,       0,       0,   0,   0
16384:                16384,      0,       2,       1,      97,   0,   0
16384:                16384,      0,      11,       0,      11,   0,   0
16384:                16384,      0,       1,       9,     164,   0,   0
16384:                16384,      0,       1,       6,      13,   0,   0
16384:                16384,      0,       1,       0,       1,   0,   0
32768:                32768,      0,       0,       4,       8,   0,   0
32768:                32768,      0,       2,       0,       2,   0,   0
32768:                32768,      0,       0,       0,       0,   0,   0
32768:                32768,      0,       0,       0,       0,   0,   0
32768:                32768,      0,      13,       0,      13,   0,   0
32768:                32768,      0,       2,       0,       2,   0,   0
32768:                32768,      0,       0,       6,      30,   0,   0
32768:                32768,      0,      12,       0,      12,   0,   0
65536:                65536,      0,       0,       6,    2062,   0,   0
65536:                65536,      0,       0,       0,       0,   0,   0
65536:                65536,      0,       0,       0,       0,   0,   0
65536:                65536,      0,       6,       1,     104,   0,   0
65536:                65536,      0,       1,       0,       1,   0,   0
65536:                65536,      0,       1,       2,      13,   0,   0
65536:                65536,      0,       0,       4,      10,   0,   0
65536:                65536,      0,       0,       0,       0,   0,   0
SLEEPQUEUE:              80,      0,     281,     246,     281,   0,   0
64 pcpu:                  8,      0,    1842,     462,    1846,   0,   0
Files:                   80,      0,     149,     751,  264053,   0,   0
filedesc0:             1104,      0,      53,      64,   16616,   0,   0
rl_entry:                40,      0,     168,     932,     168,   0,   0
TURNSTILE:              136,      0,     281,     159,     281,   0,   0
umtx pi:                 96,      0,       0,       0,       0,   0,   0
umtx_shm:                88,      0,       0,       0,       0,   0,   0
MAC labels:              40,      0,   52353,    1347,  200398,   0,   0
mac_mls:                112,      0,   52528,    1372,21106277,   0,   0
mac_biba:               112,      0,   52528,     847,21106277,   0,   0
PROC:                  1344,      0,      52,      56,   16615,   0,   0
THREAD:                1360,      0,     239,      41,    3200,   0,   0
cpuset:                  96,      0,      84,     490,     121,   0,   0
audit_record:          1280,      0,       0,       0,       0,   0,   0
mbuf_packet:            256, 1615545,      22,    1243, 1054258,   0,   0
mbuf:                   256, 1615545,    6228,    1627,20476925,   0,   0
mbuf_cluster:          2048, 757350,    7309,     109, 7108913,   0,   0
mbuf_jumbo_page:       4096, 126213,       0,      46,     300,   0,   0
mbuf_jumbo_9k:         9216,  37396,       0,       0,       0,   0,   0
mbuf_jumbo_16k:       16384,  21035,       0,       0,       0,   0,   0
NetGraph items:          72,   4123,       0,     279,      26,   0,   0
NetGraph data items:     72,   4123,       0,     527, 1775955,   0,   0
g_bio:                  376,      0,       0,    1720,  600366,   0,   0
DMAR_MAP_ENTRY:         120,      0,       0,       0,       0,   0,   0
ttyinq:                 160,      0,     255,     195,    1410,   0,   0
ttyoutq:                256,      0,     132,     303,     730,   0,   0
nvme_request:           128,      0,       0,       0,       0,   0,   0
cryptop:                 88,      0,       0,       0,       0,   0,   0
cryptodesc:              72,      0,       0,       0,       0,   0,   0
vtnet_tx_hdr:            24,      0,       0,       0,       0,   0,   0
FPU_save_area:          832,      0,       0,       0,       0,   0,   0
VNODE:                  472,      0,   51606,     282,   64870,   0,   0
VNODEPOLL:              120,      0,       4,     392,       5,   0,   0
BUF TRIE:               144,      0,    3578,   23098,   28233,   0,   0
NAMEI:                 1024,      0,       0,      80, 9028599,   0,   0
rentr:                   24,      0,       0,       0,       0,   0,   0
S VFS Cache:            108,      0,   51645,     855,   75910,   0,   0
STS VFS Cache:          148,      0,       0,       0,       0,   0,   0
L VFS Cache:            328,      0,    1418,      94,    1538,   0,   0
LTS VFS Cache:          368,      0,       0,       0,       0,   0,   0
NCLNODE:                592,      0,       0,       0,       0,   0,   0
nandfs node zone:       312,      0,       0,       0,       0,   0,   0
DIRHASH:               1024,      0,     398,      46,     519,   0,   0
fuse_ticket:            224,      0,       0,       0,       0,   0,   0
autofs_request:        5280,      0,       0,       7,      74,   0,   0
autofs_node:            208,      0,       4,     129,       4,   0,   0
Mountpoints:           2744,      0,       7,      14,      33,   0,   0
AIO:                    224,      0,       0,       0,       0,   0,   0
AIOP:                    32,      0,       0,       0,       0,   0,   0
AIOCB:                 1520,      0,       0,       0,       0,   0,   0
AIOL:                   128,      0,       0,       0,       0,   0,   0
AIOLIO:                 280,      0,       0,       0,       0,   0,   0
pipe:                   760,      0,       8,      72,    2352,   0,   0
procdesc:               136,      0,       0,       0,       0,   0,   0
ksiginfo:               112,      0,      94,     956,  416385,   0,   0
itimer:                 352,      0,       0,       0,       0,   0,   0
bridge_rtnode:           64,      0,       0,       0,       0,   0,   0
KNOTE:                  160,      0,      20,     305,   83900,   0,   0
socket:                 856, 129972,      53,      79,   38782,   0,   0
IPsec SA lft_c:          16,      0,       0,       0,       0,   0,   0
ipq:                     56,  23714,       0,     781,      29,   0,   0
udp_inpcb:              424, 129978,      20,     160,   37423,   0,   0
udpcb:                   32, 130000,      20,    1355,   37423,   0,   0
tcp_inpcb:              424, 129978,      11,     106,     138,   0,   0
tcpcb:                  920, 129972,      11,      41,     138,   0,   0
tcptw:                   88,  26010,       0,     540,      55,   0,   0
syncache:               168,  15364,       0,     253,       6,   0,   0
hostcache:               96,  15375,       0,     287,       7,   0,   0
sackhole:                32,      0,       0,       0,       0,   0,   0
tcpreass:                40,  47400,       0,       0,       0,   0,   0
sctp_ep:               1208, 129972,       0,       0,       0,   0,   0
sctp_asoc:             2400,  40000,       0,       0,       0,   0,   0
sctp_laddr:              48,  80012,       0,     830,       9,   0,   0
sctp_raddr:             728,  80000,       0,       0,       0,   0,   0
sctp_chunk:             152, 400010,       0,       0,       0,   0,   0
sctp_readq:             152, 400010,       0,       0,       0,   0,   0
sctp_stream_msg_out:    112, 400015,       0,       0,       0,   0,   0
sctp_asconf:             40, 400000,       0,       0,       0,   0,   0
sctp_asconf_ack:         48, 400060,       0,       0,       0,   0,   0
udplite_inpcb:          424, 129978,       0,       0,       0,   0,   0
ripcb:                  424, 129978,       1,     116,      88,   0,   0
unpcb:                  240, 129984,      17,     239,    1114,   0,   0
rtentry:                208,      0,      56,     191,      70,   0,   0
IPFW counters:           16,      0,      57,     967,      57,   0,   0
IPFW dynamic rule:      128,  16399,      12,    1631,   48491,   0,   0
divcb:                  424, 129978,       0,       0,       0,   0,   0
selfd:                   64,      0,     233,    1317,23093991,   0,   0
swpctrie:               144, 504873,       0,       0,       0,   0,   0
swblk:                  136, 504861,       0,       0,       0,   0,   0
FFS inode:              160,      0,   50104,     421,   62783,   0,   0
FFS1 dinode:            128,      0,       0,       0,       0,   0,   0
FFS2 dinode:            256,      0,   50104,     416,   62782,   0,   0
TMPFS dirent:            64,      0,    1298,     500,    1298,   0,   0
TMPFS node:             240,      0,    1299,     173,    1299,   0,   0
TMPFS dirent:            64,      0,     158,     896,     716,   0,   0
TMPFS node:             240,      0,     159,     273,     713,   0,   0
Received on Fri Sep 29 2017 - 07:18:44 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:13 UTC