where did all my memory go?

From: Sean McNeil <sean_at_mcneil.com>
Date: Wed, 16 Feb 2005 12:12:10 -0800
With a system built yesterday on my amd64, I had plenty of memory
showing as free when the system completely started up.  Even after
intense usage I showed lots of free memory in top.  Over night at some
point all my memory is no longer free but inactive.  Is there anything
wrong here or is this expected behavior?  ps doesn't show any serious
usage by any particular process.  Also, if disk caches or something were
taking up the memory, I would expect it to have shown a lot earlier.

Top shows the following:

last pid: 95687;  load averages:  0.04,  0.11,  0.13    up 1+02:47:04  11:58:33
182 processes: 2 running, 180 sleeping
CPU states:  6.6% user,  0.0% nice,  2.7% system,  3.9% interrupt, 86.7% idle
Mem: 512M Active, 1097M Inact, 121M Wired, 74M Cache, 214M Buf, 15M Free
Swap: 3072M Total, 3072M Free

vmstat shows:

 procs      memory      page                    disks     faults      cpu
 r b w     avm    fre  flt  re  pi  po  fr  sr ad0 ad1   in   sy  cs us sy id
 0 4 0 1069628  93252  296   0   0   0 276   7   0   0 15232 2188 4636  3  9 88

and vmstat -m shows:


        Type  InUse MemUse HighUse Requests  Size(s)
      fmtfeed     0     0K      8K     1630
       feeder    55     3K      3K    13890  32,128
         ac97     2     1K      1K        2  16,256
        mixer     1     1K      1K        1  1024
     ratefeed     1    20K     20K     1494
  mirror data     3     1K      2K    18645  64,256,512
          agp     2   129K    129K        2  32
        linux    10     1K      1K       10  64
  NTFS nthash     1   512K    512K        1
MSDOSFS mount     1   256K    256K        1
     atkbddev     2     1K      1K        2  32
        DEVFS   369   101K    101K      370  16,64,128,256
     nexusdev     2     1K      1K        2  16
          USB   107    19K     19K      129  16,32,64,128,256,512
     I/O APIC     1     2K      2K        1  2048
       USBdev     9     2K     12K       33  16,64,128,512,1024
        USBHC     2     1K      1K        3  16,128
      memdesc     1     4K      8K       26  32,4096
      entropy  1024    64K     64K     1024  64
    VM pgdata     2   129K    129K        2  128
     ppbusdev     2     1K      1K        2  256
      ATA DMA     4     1K      1K        4  256
    AD driver     4     1K      1K        4  256
      UMAHash     2    12K     14K        9  512,1024,2048,4096
    UFS mount     9    60K     60K        9  512,2048
    UFS ihash     1   512K    512K        1
  UFS dirhash  2656  1105K   1330K    15744  16,32,64,128,256,512,1024,2048
      pagedep     2   129K    181K    13658  128
     inodedep     1   512K   2376K   105788  256
       newblk     1     1K      1K   446846  64,512
    bmsafemap     0     0K      2K     6275  64
  allocdirect     0     0K    586K   304042  128
     indirdep     0     0K    338K     3275  64
   allocindir     0     0K    654K   142803  128
     freefrag     0     0K     56K   157074  64
     freeblks     0     0K    512K    57013  256
     freefile     0     0K    128K    54511  64
       diradd     0     0K    224K    87929  64
        mkdir     0     0K      9K     9840  64
       dirrem     0     0K    225K    87223  64
    newdirblk     0     0K      1K       87  32
     savedino     0     0K    520K    34555  256
     p1003.1b     1     1K      1K        1  16
  NFS srvsock     2     1K      1K        2  256
NFSV3 srvdesc     0     0K      2K    35458  32,512
   NFS daemon    69     7K      7K       69  64,512
      NFS req     0     0K      1K       36  128
     NFS hash     1   512K    512K        1
 ip6_moptions     1     1K      1K        1  32
    in6_multi    21     2K      2K       21  32,64
     syncache     1    12K     12K        1
    hostcache     1    48K     48K        1
  ip_moptions     2     1K      1K        2  256
  IpFw/IpAcct    25     5K      5K       25  64,128,2048
     in_multi     5     1K      1K        5  64
         igmp     2     1K      1K        2  32
     routetbl   196    73K     73K   194086  32,64,128,256,512
           lo     1     2K      2K        1  2048
        clone     3    12K     12K        3  4096
       ifaddr    46    19K     19K       46  32,64,256,512,4096
  ether_multi    92     5K      5K       92  16,32,64
          BPF     8    10K     10K        8  128,512,4096
  ATA generic    12     4K      4K       12  16,512
        mount   124    17K     18K      428  16,32,64,128,256,1024,2048
       vnodes     4     1K      1K    15887  32,256
  Export Host     3     2K      2K        3  512
cluster_save buffer     0     0K      1K    38456  64,128
     vfscache     1  1024K   1024K        1
   BIO buffer  5435 10870K  10940K   236537  2048
      acpidev    54     4K      4K       54  64
       soname   240    28K     30K   560515  16,32,64,128
          pcb   158    13K     14K    19609  16,32,64,128,4096
    mbuf tags     0     0K      1K   387337  32,128
     mbextcnt     0     0K      1K      854  16
         ptys    10     3K      3K       10  256
         ttys  2303   315K    329K    15467  128,1024
          shm    24    62K     72K     1633  2048
          sem     4    60K     60K        4  4096
          msg     4    30K     30K        4  2048,4096
     ioctlops     0     0K      4K 13704684  16,32,64,128,256,512,1024,4096
          iov     0     0K      1K  3037291  16,64,128,256,512,1024
       Unitno     8     1K      1K        9  32
   turnstiles   313    40K     40K      313  128
    taskqueue     6     1K      1K        6  128
     acpitask     0     0K      1K        2  16,64
 sleep queues   313    20K     20K      313  64
         sbuf     0     0K     37K     2469  16,32,64,128,256,512,1024,2048,4096
      acpipwr     1     1K      1K        1  64
         rman   159    20K     20K      573  16,128
       acpica  1871   194K    195K    17558  16,32,64,128,256,512,1024
         kobj   146   584K    596K    11596  4096
 eventhandler    38     5K      5K       38  64,256
      devstat    18    37K     37K       18  32,4096
          bus   714    59K    166K     3582  16,32,64,128,256,1024
       bus-sc    69   107K    203K     1351  16,32,64,128,256,512,1024,2048,4096
         sbp    96    14K     14K       96  32,256
         SWAP     2   413K    413K        2  64
       sysctl     0     0K      1K   257517  16,32,64
    sysctloid  2603   126K    126K     2653  16,32,64,128
    sysctltmp     0     0K      1K    24292  16,32,64,128
       plimit    35     9K     10K    12769  256
      uidinfo    14     3K      3K      428  64,2048
         cred   172    43K     45K  3054902  256
       if_fwe   128    40K     40K      128  256
         pgrp    72     9K     10K     1291  128
      session    65    17K     18K      872  256
         proc     2    16K     16K        2
      subproc   435   914K   1026K   195715  512,4096
     mtx_pool     1    12K     12K        1
     PCI Link    16     2K      2K       16  64,128
       module   231    29K     29K      231  128
      acpisem    19     3K      3K       19  128
       devbuf  5323  8315K   8571K     5992  16,32,64,128,256,512,1024,2048,4096
         temp    85    48K     48K  1183962  16,32,64,128,256,512,1024,2048,4096
       ip6opt     2     1K      1K        2  256
       ip6ndp     8     2K      2K       11  64,128,256
        lockf    41     6K      6K    47860  128
       linker    93    13K     13K      111  16,32,64,128,256,512
     firewire    29    44K     44K       31  16,64,128,512,1024,2048,4096
       KTRACE   100    13K     13K      100  128
      fw_xfer     0     0K      1K        1  256
      ithread    60    12K     12K       60  128,256
       zombie     0     0K      3K   195280  256
    proc-args   114     9K     10K   160802  16,32,64,128,256
       kqueue     0     0K     12K    20824  256,2048
         kenv    55     6K      6K       56  16,32,64,4096
    file desc   307   333K    341K   204062  16,32,64,128,256,512,1024,2048,4096
        sigio     2     1K      1K       10  64
         cdev   148    74K     74K      157  512
   ACD driver     2     4K      4K        2  2048
ATA CAM transport     4     1K      5K       88  64,256,1024
    AR driver     1     1K      3K        9  128,256,2048
  ISOFS mount     1   512K    512K        1
       isadev     7     1K      1K        7  64
         GEOM   163    30K     56K     1004  16,32,64,128,256,512,1024,2048,4096
  pfs_vncache     0     0K      1K      324  64
   pfs_fileno     2    40K     40K        2
    pfs_nodes    49     7K      7K       49  128

and finally vmstat -z shows:

ITEM            SIZE     LIMIT     USED    FREE  REQUESTS

SWAPMETA:        288,   116519,      0,      0,        0
FFS2 dinode:     256,        0,  90328,     47,  5119957
FFS1 dinode:     128,        0,      0,      0,        0
FFS inode:       208,        0,  90328,     50,  5119957
IPFW dynamic:     80,        0,    194,    391,    12353
divcb:           304,    12336,      1,     23,        1
rtentry:         264,        0,     31,     25,       81
unpcb:           192,    25600,    426,     74,    45295
ripcb:           304,    12336,      3,     21,       10
sackhole:         24,        0,      0,    252,       26
tcpreass:         40,     1680,      0,    168,     5400
hostcache:       136,    15372,     38,     46,      466
syncache:        128,    15370,      0,     58,     5751
tcptw:            80,     2475,      3,    132,     4415
tcpcb:           736,    12330,    193,     32,    11508
inpcb:           304,    12336,    196,     68,    11508
udpcb:           304,    12336,     58,     38,    15631
socket:          568,    12334,    752,     95,    72452
KNOTE:           120,        0,      0,     62,    21596
PIPE:            720,        0,    194,     36,    99726
DIRHASH:        1024,        0,   1062,    346,    18503
NFSNODE:         680,        0,      2,     10,        2
NFSMOUNT:        576,        0,      2,     12,        2
L VFS Cache:     327,        0,    565,    803,   104791
S VFS Cache:     104,        0,  95620,    536,  5858234
NAMEI:          1024,        0,      0,     20, 44514947
VNODEPOLL:       128,        0,      1,     57,        1
VNODE:           512,        0,  90398,     14,    90398
ata_request:     344,        0,      0,     44,  1205278
g_bio:           216,        0,      0,   1944,  4741157
MbufClust:      2048,    25600,   1536,      6,    13308
Mbuf:            256,        0,   1538,    127,  6041041
Packet:          256,        0,   1472,    193,  3353442
VMSPACE:         544,        0,    127,     48,   195884
UPCALL:           88,        0,     34,     42,      255
KSEGRP:          136,        0,    252,     34,      364
TID:             144,        0,      1,     51,        1
THREAD:          608,        0,    299,     13,  6232098
PROC:            800,        0,    189,     46,   195498
Files:           120,        0,   1430,    213,  4033993
4096:           4096,        0,    638,     39,   215187
2048:           2048,        0,   5488,    134,   249953
1024:           1024,        0,    135,     85,  1002937
512:             512,        0,   2980,    450,   251983
256:             256,        0,   1111,   1829,  3682133
128:             128,        0,   8229,   2472,   850191
64:               64,        0,   3842,   3830,  3898580
32:               32,        0,   1690,    330,   886456
16:               16,        0,   1935,    585, 14240406
DP fakepg:       120,        0,  14590,     11,    14590
PV ENTRY:         48,  1757088, 285743,  73753, 235121221
MAP ENTRY:       112,        0,  17802,   1932,  9757492
KMAP ENTRY:      112,    88011,     15,    117,    11943
MAP:             352,        0,      7,     15,        5
VM OBJECT:       224,        0,  88858,   8739,  4165303
128 Bucket:     1048,        0,    525,      0,        0
64 Bucket:       536,        0,     28,     35,        0
32 Bucket:       280,        0,     28,     28,        0
16 Bucket:       152,        0,     29,     21,        0
UMA Hash:        256,        0,      3,     12,        0
UMA RCntSlab:    128,        0,    771,     12,        0
UMA Slabs:       128,        0,   3976,     26,        0
UMA Zones:       152,        0,     62,     10,        0
UMA Kegs:        232,        0,     62,     13,        0
Received on Wed Feb 16 2005 - 19:12:12 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:38:28 UTC