Re: DTrace for FreeBSD - fbt output

From: Robert Watson <rwatson_at_FreeBSD.org> Date: Wed, 14 Jun 2006 10:39:14 +0100 (BST) · This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:38:57 UTC

On Tue, 13 Jun 2006, John Birrell wrote:

> On Tue, Jun 13, 2006 at 02:15:43AM +0000, John Birrell wrote:
>> With the development as it stands at the moment, take care using the FBT
>> provider because you can easily cause the system to go kaboom. I'm still
>> trying to track down the problems there. It's not in FBT itself -- just
>> the fact that the DTrace probe context isn't allowed to call anything that
>> FBT can instrument. If that happens you will either get a reboot or a
>> double fault will leave you in kdb. I recommend only enabling a few FBT
>> probes at a time just so you know which ones could cause a fault. There is
>> no point telling me that you enabled fbt::: and the system went kaboom!
>
> With the FBT provider as it now stands and using this script:
>
> fbt:::entry
> {
>        _at_[probefunc] = count();
> }
>
> the output after a buildworld is listed below. Check out the number of calls 
> to critical_enter and critical_exit (which are listed at the bottom)! And 
> for comparison, check the hardclock() count which relects 1000 Hz.

I don't suppose it's possible to get stack traces or fractional stack traces 
(last 2-3 frames)?  There are a number of comon consumers of critical 
sections, but the number does seem inordinately high, especially with respect 
to the number of calls of the common consumers.  Critical sections are 
acquired during access to per-cpu caches in UMA, and for per-cpu stats in 
malloc(9), as well as in context switches and in interrupts.

BTW, it looks like the below is running with invariants turned on?  It may be 
more interesting to run the below without invariants, since invariants 
significantly changes the behavior of the memory allocation paths due to 
memory trashing and additional bookkeeping.

Robert N M Watson
Computer Laboratory
Universty of Cambridge

>
>
>  cnioctl                                                           1
>  cr_cansignal                                                      1
>  dead_getwritemount                                                1
>  devfs_delete                                                      1
>  devfs_free                                                        1
>  devfs_statfs                                                      1
>  fpathconf                                                         1
>  ip6_ctloutput                                                     1
>  kern_sigtimedwait                                                 1
>  kill                                                              1
>  p_cansignal                                                       1
>  pfs_statfs                                                        1
>  pps_ioctl                                                         1
>  rip6_attach                                                       1
>  rip6_ctloutput                                                    1
>  rip6_detach                                                       1
>  rip_attach                                                        1
>  rip_ctloutput                                                     1
>  rip_detach                                                        1
>  rip_pcbdetach                                                     1
>  sigqueue_move                                                     1
>  sigtimedwait                                                      1
>  startup_alloc                                                     1
>  sysctl_kern_boottime                                              1
>  sysctl_kern_cp_time                                               1
>  zero_read                                                         1
>  accept                                                            2
>  accept1                                                           2
>  chflags                                                           2
>  cloneuio                                                          2
>  cnclose                                                           2
>  cnwrite                                                           2
>  devclass_get_device                                               2
>  devclass_get_softc                                                2
>  faithprefix                                                       2
>  fchown                                                            2
>  fgetown                                                           2
>  getpriority                                                       2
>  getsockname                                                       2
>  getsockname1                                                      2
>  icmp6_input                                                       2
>  icmp6_rip6_input                                                  2
>  in6_cksum                                                         2
>  in6_ifawithifp                                                    2
>  in_setsockaddr                                                    2
>  ip6_addaux                                                        2
>  ip6_delaux                                                        2
>  ip6_findaux                                                       2
>  ip6_getdstifaddr                                                  2
>  ip6_hopopts_input                                                 2
>  ip6_input                                                         2
>  ip6_process_hopopts                                               2
>  ip6_setdstifaddr                                                  2
>  log_console                                                       2
>  m_tag_copy                                                        2
>  memopen                                                           2
>  mld6_input                                                        2
>  ndflush                                                           2
>  pffindproto                                                       2
>  pty_create_slave                                                  2
>  ptyinit                                                           2
>  revoke                                                            2
>  sa6_recoverscope                                                  2
>  scclose                                                           2
>  setfflags                                                         2
>  siobusycheck                                                      2
>  siopoll                                                           2
>  soaccept                                                          2
>  sonewconn                                                         2
>  syncache_add                                                      2
>  syncache_drop                                                     2
>  syncache_expand                                                   2
>  syncache_free                                                     2
>  syncache_insert                                                   2
>  syncache_respond                                                  2
>  syncache_socket                                                   2
>  syncookie_generate                                                2
>  sysctl_kern_msgbuf                                                2
>  sysctl_vfs_conflist                                               2
>  sysctl_vm_swap_info                                               2
>  tcp_hc_get                                                        2
>  tcp_mss                                                           2
>  tcp_reass                                                         2
>  tcp_sockaddr                                                      2
>  tcp_usr_accept                                                    2
>  tty_close                                                         2
>  ttyalloc                                                          2
>  ttylclose                                                         2
>  ttyrel                                                            2
>  vcount                                                            2
>  wakeup_one                                                        2
>  cnopen                                                            3
>  genkbd_commonioctl                                                3
>  ifconf                                                            3
>  in_pcbbind                                                        3
>  kbdmux_ioctl                                                      3
>  kern_sigsuspend                                                   3
>  malloc_type_allocated                                             3
>  sbuf_data                                                         3
>  sbuf_delete                                                       3
>  sbuf_finish                                                       3
>  sbuf_new                                                          3
>  sc_get_softc                                                      3
>  scdevtounit                                                       3
>  scopen                                                            3
>  sigsuspend                                                        3
>  soisconnecting                                                    3
>  tcp_connect                                                       3
>  tcp_drop                                                          3
>  tcp_new_isn                                                       3
>  tcp_respond                                                       3
>  tcp_usr_connect                                                   3
>  ttydtrwaitsleep                                                   3
>  ttyopen                                                           3
>  uma_large_free                                                    3
>  uma_large_malloc                                                  3
>  vgone                                                             3
>  comstart                                                          4
>  devfs_alloc                                                       4
>  devfs_create                                                      4
>  devfs_newdirent                                                   4
>  devfs_rules_apply                                                 4
>  getlogin                                                          4
>  getsockopt                                                        4
>  giant_close                                                       4
>  hash_alloc                                                        4
>  hash_expand                                                       4
>  hash_free                                                         4
>  ip_srcroute                                                       4
>  kern_getsockopt                                                   4
>  kern_setsockopt                                                   4
>  logread                                                           4
>  m_tag_alloc                                                       4
>  m_tag_delete                                                      4
>  m_tag_free_default                                                4
>  make_dev_cred                                                     4
>  make_dev_credv                                                    4
>  msgbuf_getbytes                                                   4
>  newdev                                                            4
>  nlminfo_release                                                   4
>  pfs_reclaim                                                       4
>  pfs_vncache_free                                                  4
>  setsockopt                                                        4
>  sogetopt                                                          4
>  sooptcopyin                                                       4
>  sosetopt                                                          4
>  syncache_lookup                                                   4
>  termioschars                                                      4
>  ttyinitmode                                                       4
>  vsnrprintf                                                        4
>  MD5Final                                                          5
>  MD5Init                                                           5
>  MD5Pad                                                            5
>  MD5Transform                                                      5
>  ptcopen                                                           5
>  tcp_hc_getmtu                                                     5
>  tcp_mssopt                                                        5
>  vslock                                                            5
>  vsunlock                                                          5
>  cblock_free_cblocks                                               6
>  clist_free_cblocks                                                6
>  getfsstat                                                         6
>  if_simloop                                                        6
>  in6_setscope                                                      6
>  in_addroute                                                       6
>  ip_ctloutput                                                      6
>  kern_getfsstat                                                    6
>  looutput                                                          6
>  netisr_queue                                                      6
>  pfs_close                                                         6
>  pfs_open                                                          6
>  pfs_read                                                          6
>  proc_rwmem                                                        6
>  procfs_close                                                      6
>  procfs_doprocmem                                                  6
>  ptcioctl                                                          6
>  rn_addroute                                                       6
>  rn_delete                                                         6
>  rn_insert                                                         6
>  rn_newpair                                                        6
>  rt_missmsg                                                        6
>  rt_setgate                                                        6
>  tcp_ctloutput                                                     6
>  uiomove_fromphys                                                  6
>  vrefcnt                                                           6
>  VOP_LINK_APV                                                      7
>  can_hardlink                                                      7
>  dev_ref                                                           7
>  devtoname                                                         7
>  getgroups                                                         7
>  kern_link                                                         7
>  link                                                              7
>  obj_alloc                                                         7
>  pgsignal                                                          7
>  ptsopen                                                           7
>  tcp_disconnect                                                    7
>  tcp_hc_lookup                                                     7
>  tcp_maxmtu                                                        7
>  ttyref                                                            7
>  udp6_attach                                                       7
>  udp6_detach                                                       7
>  ufs_link                                                          7
>  vcnt                                                              7
>  vop_link_post                                                     7
>  cru2x                                                             8
>  in6_clearscope                                                    8
>  in6_pcbdetach                                                     8
>  in6_pcbfree                                                       8
>  in_setpeeraddr                                                    8
>  ip6_freemoptions                                                  8
>  ip6_freepcbopts                                                   8
>  pfs_root                                                          8
>  siointr                                                           8
>  sysctl_kern_proc                                                  8
>  sysctl_wire_old_buffer                                            8
>  tcp_peeraddr                                                      8
>  cn_devopen                                                        9
>  icmp_error                                                        9
>  icmp_reflect                                                      9
>  icmp_send                                                         9
>  in_canforward                                                     9
>  m_align                                                           9
>  pipe_poll                                                         9
>  ptsstop                                                           9
>  scread                                                            9
>  snprintf                                                          9
>  ttyread                                                           9
>  draw_txtcharcursor                                               10
>  in6_addrscope                                                    10
>  in_pcbdrop                                                       10
>  pipe_zone_init                                                   10
>  sc_draw_cursor_image                                             10
>  sc_inside_cutmark                                                10
>  sc_vtb_copy                                                      10
>  sc_vtb_geta                                                      10
>  sc_vtb_getc                                                      10
>  sc_vtb_putc                                                      10
>  setugidsafety                                                    10
>  siointr1                                                         10
>  tcp_close                                                        10
>  tcp_detach                                                       10
>  tcp_discardcb                                                    10
>  tcp_free_sackholes                                               10
>  tcp_usr_detach                                                   10
>  vga_txtcursor                                                    10
>  vga_txtdraw                                                      10
>  devfs_populate_loop                                              11
>  kernacc                                                          11
>  memrw                                                            11
>  pmap_kenter                                                      11
>  arp_rtrequest                                                    12
>  badport_bandlim                                                  12
>  dev2unit                                                         12
>  dupfdopen                                                        12
>  in_sockaddr                                                      12
>  minor2unit                                                       12
>  pfs_lookup                                                       12
>  ppsratecheck                                                     12
>  rn_search                                                        12
>  rtrequest                                                        12
>  rtrequest1                                                       12
>  sysctl_machdep_adjkerntz                                         12
>  tcp_attach                                                       12
>  tcp_newtcpcb                                                     12
>  tcp_usr_attach                                                   12
>  ttymodem                                                         12
>  arc4_randomstir                                                  13
>  link_elf_symbol_values                                           13
>  nottystop                                                        13
>  sc_touch_scrn_saver                                              13
>  scterm_input                                                     13
>  tty_open                                                         13
>  read_random                                                      14
>  vfsconf2x                                                        14
>  ifhwioctl                                                        15
>  ifunit                                                           15
>  m_tag_delete_nonpersistent                                       15
>  getpeername                                                      16
>  getpeername1                                                     16
>  orphanpg                                                         16
>  kqueue                                                           17
>  kqueue_close                                                     17
>  vop_stdbmap                                                      17
>  bpf_clone                                                        18
>  ctty_clone                                                       18
>  devfs_fqpn                                                       18
>  ifioctl                                                          18
>  kevent_copyout                                                   18
>  knote_enqueue                                                    18
>  kqueue_wakeup                                                    18
>  procfs_attr                                                      18
>  sc_puts                                                          18
>  scterm_puts                                                      18
>  tunclone                                                         18
>  arprequest                                                       19
>  kern_statfs                                                      19
>  nd6_slowtimo                                                     19
>  statfs                                                           19
>  dev_refl                                                         20
>  inferior                                                         20
>  pfs_vncache_alloc                                                20
>  ttywrite                                                         20
>  sbuf_len                                                         21
>  sbuf_overflowed                                                  21
>  scparam                                                          21
>  dev_stdclone                                                     22
>  kbdmux_intr                                                      22
>  kbdmux_kbd_intr                                                  22
>  rn_satisfies_leaf                                                22
>  rn_search_m                                                      22
>  sckbdevent                                                       22
>  taskqueue_enqueue                                                22
>  taskqueue_run                                                    22
>  taskqueue_swi_giant_enqueue                                      22
>  taskqueue_swi_giant_run                                          22
>  ttyflush                                                         22
>  dev_rel                                                          24
>  procfs_candebug                                                  24
>  atkbd_intr                                                       26
>  atkbd_read_char                                                  26
>  atkbdintr                                                        26
>  devfs_reclaim                                                    26
>  genkbd_keyaction                                                 26
>  kbdmux_kbd_event                                                 26
>  read_kbd_data_no_wait                                            26
>  MD5Update                                                        27
>  in_pcbdisconnect                                                 28
>  knlist_add                                                       28
>  knlist_remove                                                    28
>  knlist_remove_kq                                                 28
>  knote_attach                                                     28
>  knote_drop                                                       28
>  soo_kqfilter                                                     28
>  turnstile_adjust                                                 28
>  udp_connect                                                      28
>  udp_disconnect                                                   28
>  kern_kevent                                                      29
>  kevent                                                           29
>  kevent_copyin                                                    29
>  kqueue_expand                                                    29
>  kqueue_fo_find                                                   29
>  kqueue_fo_release                                                29
>  kqueue_register                                                  29
>  kqueue_release                                                   29
>  kqueue_scan                                                      29
>  in_pcbconnect                                                    30
>  is_unsafe                                                        30
>  p_candebug                                                       30
>  pfs_access                                                       30
>  soo_ioctl                                                        30
>  in_pcbbind_setup                                                 31
>  in_pcblookup_local                                               31
>  ip_rtaddr                                                        31
>  pfs_getattr                                                      31
>  scstart                                                          31
>  turnstile_release                                                31
>  tcp_timer_delack                                                 32
>  bufsync                                                          33
>  in_pcbconnect_setup                                              33
>  in_pcbinshash                                                    33
>  giant_open                                                       35
>  kbdmux_read_char                                                 35
>  scgetc                                                           35
>  prison_canseemount                                               36
>  pty_clone                                                        36
>  sbuf_bcat                                                        36
>  sc_remove_cutmarking                                             36
>  sc_vtb_append                                                    36
>  sc_vtb_delete                                                    36
>  pgadjustjobc                                                     38
>  trash_fini                                                       38
>  udp_attach                                                       38
>  udp_detach                                                       38
>  kldsym                                                           39
>  setpgid                                                          42
>  mtx_pool_find                                                    45
>  pmap_pinit                                                       45
>  sc_hist_ioctl                                                    45
>  sc_mouse_ioctl                                                   45
>  sc_vid_ioctl                                                     45
>  scioctl                                                          45
>  scterm_ioctl                                                     45
>  sx_init                                                          45
>  vop_einval                                                       45
>  kqueue_aquire                                                    46
>  ttypend                                                          46
>  in_pcbdetach                                                     49
>  in_pcbfree                                                       49
>  ufsdirhash_dirtrunc                                              49
>  in_pcbremlists                                                   57
>  ip_freemoptions                                                  57
>  knote_alloc                                                      58
>  knote_free                                                       58
>  in_pcballoc                                                      59
>  prison_ip                                                        59
>  in_pcbrehash                                                     61
>  arc4rand                                                         62
>  arc4random                                                       62
>  knlist_empty                                                     63
>  ksiginfo_alloc                                                   63
>  proc_init                                                        63
>  proc_linkup                                                      63
>  pstats_alloc                                                     63
>  sched_newproc                                                    63
>  pfs_visible                                                      67
>  sc_get_stat                                                      79
>  cpu_thread_setup                                                 80
>  sleepq_alloc                                                     80
>  turnstile_alloc                                                  80
>  umtxq_alloc                                                      80
>  cv_init                                                          90
>  ufsdirhash_move                                                  91
>  bucket_zone_drain                                                92
>  frag6_drain                                                      92
>  in_rtqdrain                                                      92
>  ip_drain                                                         92
>  mb_reclaim                                                       92
>  uma_reclaim                                                      92
>  kse_wakeup                                                       93
>  in_clsroute                                                      98
>  rtfree                                                           98
>  arpintr                                                         100
>  in_arpinput                                                     100
>  msglogchar                                                      108
>  turnstile_adjust_thread                                         109
>  pagedaemon_wakeup                                               111
>  in6_rtqtimo                                                     114
>  in_rtqtimo                                                      114
>  msgbuf_addchar                                                  116
>  ttywait                                                         116
>  change_rgid                                                     121
>  catq                                                            123
>  change_ruid                                                     123
>  kmem_alloc_nofault                                              125
>  vlrureclaim                                                     126
>  change_svgid                                                    128
>  change_svuid                                                    131
>  ttsetwater                                                      137
>  arplookup                                                       150
>  null_ioctl                                                      166
>  tcp_drain                                                       184
>  devfs_setattr                                                   194
>  minor                                                           207
>  getdtablesize                                                   210
>  trash_init                                                      216
>  arptimer                                                        229
>  tcp_hc_purge                                                    229
>  prison_enforce_statfs                                           241
>  elf_hash                                                        247
>  link_elf_lookup_symbol                                          247
>  sc_vtb_pointer                                                  248
>  turnstile_first_waiter                                          258
>  uio_yield                                                       362
>  change_egid                                                     363
>  change_euid                                                     364
>  setlogin                                                        380
>  uipc_detach                                                     381
>  uipc_disconnect                                                 381
>  unp_detach                                                      381
>  unp_disconnect                                                  381
>  unp_dispose                                                     381
>  setsid                                                          382
>  uipc_attach                                                     383
>  unp_attach                                                      383
>  unp_connect2                                                    383
>  sysctl_kern_ps_strings                                          386
>  uipc_connect                                                    386
>  unp_connect                                                     386
>  setgid                                                          387
>  donice                                                          388
>  p_cansched                                                      388
>  sched_nice                                                      388
>  setpriority                                                     388
>  pgdelete                                                        394
>  sessrele                                                        394
>  setgroups                                                       397
>  setuid                                                          400
>  soisdisconnected                                                401
>  doenterpgrp                                                     404
>  enterpgrp                                                       404
>  tcp_update_sack_list                                            405
>  sodisconnect                                                    409
>  cblock_alloc_cblocks                                            411
>  clist_alloc_cblocks                                             411
>  soisconnected                                                   413
>  connect                                                         417
>  getsockaddr                                                     417
>  kern_connect                                                    417
>  soconnect                                                       417
>  pffindtype                                                      438
>  sbrelease                                                       438
>  socantrcvmore_locked                                            438
>  socantsendmore_locked                                           438
>  soclose                                                         438
>  sodealloc                                                       438
>  sofree                                                          438
>  soo_close                                                       438
>  sorflush                                                        438
>  socket                                                          440
>  socreate                                                        440
>  soalloc                                                         442
>  soreserve                                                       442
>  sysctl_hostname                                                 454
>  sysctl_out_proc                                                 460
>  pipespace                                                       464
>  sched_pctcpu                                                    470
>  sc_vtb_putchar                                                  486
>  ufsdirhash_newblk                                               486
>  bucket_drain                                                    510
>  msgbuf_peekbytes                                                516
>  calccru                                                         530
>  ufsdirhash_free                                                 560
>  sysctl_vm_loadavg                                               602
>  cpu_set_fork_handler                                            608
>  kthread_create                                                  608
>  kthread_exit                                                    608
>  ksprintn                                                        614
>  vsnprintf                                                       617
>  sysctl_hw_physmem                                               619
>  sysctl_hw_usermem                                               619
>  kvprintf                                                        621
>  ufs_chown                                                       631
>  chown                                                           633
>  kern_chown                                                      633
>  setfown                                                         635
>  ffs_indirtrunc                                                  678
>  VOP_IOCTL_APV                                                   679
>  vn_ioctl                                                        679
>  vop_enotty                                                      679
>  proc_reparent                                                   847
>  trypbuf                                                         869
>  sbflush_locked                                                  876
>  sbrelease_locked                                                876
>  pgfind                                                          880
>  bucket_free                                                     881
>  ksegrp_free                                                     887
>  ksegrp_stash                                                    887
>  ksegrp_unlink                                                   887
>  sbreserve_locked                                                888
>  sigpending                                                      888
>  sysctl_kse_virtual_cpu                                          888
>  ksegrp_alloc                                                    951
>  ksegrp_ctor                                                     951
>  ksegrp_link                                                     951
>  sched_init_concurrency                                          951
>  fchmod                                                          966
>  sysctl_kern_proc_args                                           966
>  getutimes                                                       967
>  kern_utimes                                                     967
>  setutimes                                                       967
>  utimes                                                          967
>  mtrash_fini                                                     968
>  ptcwrite                                                        970
>  soo_read                                                       1009
>  tcp_usr_rcvd                                                   1009
>  setegid                                                        1013
>  seteuid                                                        1013
>  in_rtqkill                                                     1017
>  in6_rtqkill                                                    1026
>  ptsread                                                        1078
>  ttread                                                         1087
>  cluster_rbuild                                                 1103
>  ttyecho                                                        1115
>  ttyinput                                                       1115
>  uipc_send                                                      1137
>  in6_mtutimo                                                    1142
>  kern_recvit                                                    1155
>  recvfrom                                                       1155
>  recvit                                                         1155
>  sodupsockaddr                                                  1155
>  kern_sendit                                                    1185
>  sendit                                                         1185
>  sendto                                                         1185
>  fsync                                                          1191
>  softdep_sync_metadata                                          1191
>  ttwakeup                                                       1232
>  VOP_SYMLINK_APV                                                1233
>  eaccess                                                        1233
>  kern_eaccess                                                   1233
>  kern_symlink                                                   1233
>  symlink                                                        1233
>  ufs_symlink                                                    1233
>  vop_symlink_post                                               1233
>  generator_gate                                                 1237
>  sysctl_handle_long                                             1242
>  devfs_readlink                                                 1255
>  flock                                                          1262
>  ttysleep                                                       1297
>  random_close                                                   1365
>  random_read                                                    1365
>  random_yarrow_read                                             1379
>  sleepq_signal                                                  1461
>  rn_walktree                                                    1462
>  VOP_MKDIR_APV                                                  1465
>  VOP_RMDIR_APV                                                  1465
>  ffs_dirpref                                                    1465
>  kern_rmdir                                                     1465
>  rmdir                                                          1465
>  ufs_dirempty                                                   1465
>  ufs_mkdir                                                      1465
>  ufs_rmdir                                                      1465
>  vop_mkdir_post                                                 1465
>  vop_rmdir_post                                                 1465
>  cv_wait                                                        1474
>  cv_wait_unlock                                                 1474
>  cr_cansee                                                      1488
>  p_cansee                                                       1488
>  pfind                                                          1491
>  ffs_fsync                                                      1506
>  cr_seeothergids                                                1519
>  cr_seeotheruids                                                1519
>  kern_nanosleep                                                 1519
>  nanosleep                                                      1519
>  null_write                                                     1588
>  setsugid                                                       1607
>  exec_shell_imgact                                              1627
>  copyinuio                                                      1656
>  writev                                                         1656
>  devfs_close_f                                                  1677
>  devfs_close                                                    1680
>  count_dev                                                      1681
>  devfs_open                                                     1706
>  nullop                                                         1730
>  chgsbsize                                                      1764
>  upcall_free                                                    1774
>  upcall_stash                                                   1774
>  upcall_unlink                                                  1774
>  kse_create                                                     1776
>  upcall_alloc                                                   1776
>  upcall_link                                                    1776
>  getc                                                           1777
>  devfs_root                                                     1821
>  lf_clearlock                                                   1849
>  lf_getblock                                                    1849
>  lf_setlock                                                     1849
>  lf_wakelock                                                    1849
>  ffs_oldfscompat_write                                          1918
>  ffs_sbupdate                                                   1918
>  pbgetbo                                                        1931
>  pbrelbo                                                        1931
>  vnode_pager_addr                                               1931
>  vnode_pager_generic_getpages                                   1931
>  sysctl_handle_string                                           2086
>  name2oid                                                       2106
>  sysctl_sysctl_name2oid                                         2106
>  chmod                                                          2157
>  kern_chmod                                                     2157
>  pipe_ioctl                                                     2191
>  tcp_clean_sackreport                                           2208
>  realitexpire                                                   2284
>  cv_timedwait_sig                                               2288
>  timevalsub                                                     2311
>  bucket_zone_lookup                                             2397
>  uma_zalloc_internal                                            2737
>  kern_setitimer                                                 2739
>  setitimer                                                      2739
>  getsock                                                        2785
>  bfreekva                                                       2841
>  devfs_fsync                                                    2854
>  vop_stdfsync                                                   2854
>  psignal                                                        2909
>  mtrash_init                                                    2967
>  ffs_backgroundwritedone                                        3029
>  geteblk                                                        3029
>  getnanouptime                                                  3039
>  ttnread                                                        3064
>  ttypoll                                                        3064
>  devfs_find                                                     3086
>  devfs_lookup                                                   3086
>  devfs_lookupx                                                  3086
>  maybe_preempt_in_ksegrp                                        3086
>  devfs_populate                                                 3093
>  tty_gettp                                                      3093
>  ufs_chmod                                                      3121
>  setfmode                                                       3123
>  madvise                                                        3135
>  kern_preadv                                                    3202
>  pread                                                          3202
>  cred_update_thread                                             3211
>  cv_signal                                                      3395
>  VOP_REALLOCBLKS_APV                                            3419
>  cluster_collectbufs                                            3419
>  ffs_reallocblks                                                3419
>  ffs_reallocblks_ufs2                                           3419
>  uma_timeout                                                    3426
>  ffs_clusteralloc                                               3448
>  kse_GC                                                         3453
>  ufs_pathconf                                                   3498
>  bucket_enable                                                  3518
>  zone_foreach                                                   3518
>  kern_pathconf                                                  3542
>  pathconf                                                       3542
>  VOP_PATHCONF_APV                                               3543
>  sched_set_concurrency                                          3550
>  lf_advlock                                                     3698
>  ufs_advlock                                                    3698
>  VOP_ADVLOCK_APV                                                3706
>  vn_rdwr                                                        3717
>  kmem_free                                                      3957
>  page_free                                                      3957
>  pmap_change_wiring                                             4024
>  pmap_extract                                                   4035
>  cpu_tick_calibrate                                             4303
>  cpu_set_upcall                                                 4312
>  sbdrop                                                         4379
>  vfs_stdnosync                                                  4460
>  ffs_sync                                                       4461
>  softdep_get_depcounts                                          4461
>  logpoll                                                        4547
>  msgbuf_getcount                                                4551
>  dasendorderedtag                                               4568
>  devfs_access                                                   4792
>  devfs_allocv                                                   4896
>  snprintf_func                                                  5104
>  upcall_remove                                                  5198
>  free_unr                                                       5199
>  free_unrl                                                      5199
>  alloc_unr                                                      5263
>  alloc_unrl                                                     5267
>  bucket_cache_drain                                             5336
>  poll                                                           5346
>  pollscan                                                       5346
>  lf_findoverlap                                                 5547
>  cluster_callback                                               6659
>  pbgetvp                                                        6659
>  pbrelvp                                                        6659
>  uma_zalloc_bucket                                              6797
>  kern_pwritev                                                   6814
>  pwrite                                                         6814
>  bremfreef                                                      7130
>  collapse_unr                                                   7148
>  zone_drain                                                     7176
>  ip_fragment                                                    7192
>  getpbuf                                                        7733
>  ptsioctl                                                       7780
>  l_nullioctl                                                    7826
>  ttioctl                                                        7826
>  ttyioctl                                                       7826
>  giant_ioctl                                                    7829
>  ttyldoptim                                                     7829
>  npxdrop                                                        7833
>  optimize_unr                                                   7953
>  get_mcontext                                                   7990
>  pmap_pte                                                       8058
>  timevaladd                                                     8592
>  initpbuf                                                       8602
>  relpbuf                                                        8602
>  cpu_set_upcall_kse                                             8878
>  cluster_wbuild                                                 9275
>  umask                                                          9882
>  in6_mtuexpire                                                 10278
>  timevalfix                                                    10903
>  kmem_malloc                                                   11702
>  page_alloc                                                    11702
>  slab_zalloc                                                   11706
>  devfs_stat_f                                                  12242
>  devfs_getattr                                                 12337
>  vattr_null                                                    12337
>  kern_setrlimit                                                13244
>  lim_alloc                                                     13244
>  lim_copy                                                      13244
>  setrlimit                                                     13244
>  sync_fsync                                                    13382
>  vfs_msync                                                     13382
>  vop_eopnotsupp                                                13382
>  arc4_randbyte                                                 13560
>  loadav                                                        13708
>  random                                                        13708
>  sched_load                                                    13708
>  kern_mkdir                                                    13718
>  mkdir                                                         13718
>  vfs_bio_set_validclean                                        14256
>  msync                                                         14978
>  ftruncate                                                     14985
>  ufsdirhash_recycle                                            15121
>  ffs_getpages                                                  15168
>  VOP_GETPAGES_APV                                              15184
>  vnode_pager_getpages                                          15184
>  ufs_dirrewrite                                                16091
>  VOP_PUTPAGES_APV                                              16137
>  vnode_pager_generic_putpages                                  16137
>  vnode_pager_putpages                                          16137
>  vop_stdputpages                                               16137
>  sync_vnode                                                    16552
>  kern_sigaltstack                                              16815
>  sigaltstack                                                   16815
>  sysctl_kern_clockrate                                         16815
>  sysctl_kern_usrstack                                          16815
>  relookup                                                      16939
>  VOP_RENAME_APV                                                16944
>  kern_rename                                                   16944
>  rename                                                        16944
>  ufs_rename                                                    16944
>  vop_rename_post                                               16944
>  vop_rename_pre                                                16944
>  change_dir                                                    17159
>  chdir                                                         17159
>  kern_chdir                                                    17159
>  sysctl_handle_opaque                                          17327
>  VOP_FSYNC_APV                                                 17775
>  getrusage                                                     18522
>  kern_getrusage                                                18522
>  calcru                                                        18915
>  ufsdirhash_add                                                18929
>  calcru1                                                       19445
>  cputick2usec                                                  19905
>  cpu_tickrate                                                  19926
>  sysctl_new_user                                               20251
>  getppid                                                       20476
>  getgid                                                        20485
>  getuid                                                        20651
>  in_localaddr                                                  20964
>  getegid                                                       21120
>  pipe                                                          22209
>  pipe_zone_ctor                                                22209
>  pipe_zone_dtor                                                22209
>  pipespace_new                                                 22673
>  ffs_syncvnode                                                 23332
>  ufsdirhash_delslot                                            23381
>  ufsdirhash_remove                                             23381
>  ufsdirhash_findslot                                           23472
>  vn_writechk                                                   23760
>  vfs_bio_clcheck                                               24210
>  waitrunningbufspace                                           24396
>  dup2                                                          24942
>  ufsdirhash_enduseful                                          25509
>  ufsdirhash_findfree                                           26231
>  ufsdirhash_getprev                                            28634
>  VOP_REMOVE_APV                                                29679
>  ufs_remove                                                    29679
>  vop_remove_post                                               29679
>  kern_unlink                                                   30289
>  unlink                                                        30289
>  getpgrp                                                       30855
>  dup                                                           34509
>  bawrite                                                       34513
>  pmap_zero_page_area                                           36654
>  m_copydata                                                    37686
>  pipe_stat                                                     39728
>  elf32_load_file                                               40259
>  ufsdirhash_checkblock                                         42310
>  kern___getcwd                                                 42386
>  vn_fullpath1                                                  42386
>  pipe_close                                                    44418
>  pipe_create                                                   44418
>  pipeclose                                                     44418
>  VOP_CREATE_APV                                                44538
>  ufs_create                                                    44538
>  vop_create_post                                               44538
>  funsetown                                                     44875
>  ufs_makeinode                                                 45771
>  vnode_pager_haspage                                           46347
>  vtruncbuf                                                     46733
>  vfs_bio_awrite                                                46929
>  skpc                                                          47150
>  ffs_freefile                                                  47228
>  ffs_vfree                                                     47228
>  vrecycle                                                      47228
>  ffs_nodealloccg                                               47236
>  ffs_valloc                                                    47236
>  uma_zone_slab                                                 47372
>  zero_init                                                     47879
>  random_yarrow_unblock                                         47884
>  reseed                                                        47884
>  ufs_dirremove                                                 48083
>  ufs_direnter                                                  48091
>  ufs_makedirentry                                              48091
>  rijndael_cipherInit                                           49121
>  rijndael_makeKey                                              49121
>  yarrow_encrypt_init                                           49121
>  geteuid                                                       49352
>  ffs_fragextend                                                51281
>  ffs_realloccg                                                 51281
>  ufs_bmap                                                      51533
>  VOP_BMAP_APV                                                  51550
>  npxsetregs                                                    51731
>  cluster_write                                                 52805
>  vfork                                                         53899
>  set_fpcontext                                                 55953
>  sigreturn                                                     55953
>  postsig                                                       55957
>  sendsig                                                       55957
>  sigqueue_get                                                  55958
>  issignal                                                      55962
>  tdsigwakeup                                                   55965
>  sigqueue_add                                                  55966
>  sleepq_abort                                                  56537
>  updatepri                                                     57070
>  m_fixhdr                                                      57971
>  tcp_xmit_bandwidth_limit                                      58205
>  tcp_xmit_timer                                                58205
>  sbdrop_locked                                                 58606
>  tcpip_fillheaders                                             58696
>  tcp_dooptions                                                 58806
>  tcp_input                                                     58809
>  rijndael_blockEncrypt                                         61495
>  yarrow_encrypt                                                61495
>  vn_syncer_add_to_worklist                                     61950
>  get_fpcontext                                                 63947
>  npxformat                                                     63947
>  npxgetregs                                                    63947
>  ptswrite                                                      64093
>  ttwrite                                                       64113
>  giant_write                                                   65085
>  ffs_truncate                                                  65298
>  sbappendstream                                                65364
>  soo_write                                                     65364
>  ptcread                                                       65383
>  tcp_usr_send                                                  65383
>  ttwwakeup                                                     65429
>  sbappendstream_locked                                         66362
>  sbcompress                                                    66362
>  ureadc                                                        66469
>  giant_read                                                    66485
>  sosend                                                        66520
>  sosend_copyin                                                 66549
>  devfs_write_f                                                 66671
>  knlist_mtx_lock                                               66696
>  knlist_mtx_unlock                                             66696
>  do_dup                                                        66781
>  pipe_free_kmem                                                67091
>  cv_wait_sig                                                   67377
>  getpid                                                        67433
>  devfs_read_f                                                  67907
>  bioq_takefirst                                                68310
>  schedcpu                                                      68310
>  if_slowtimo                                                   68311
>  ipport_tick                                                   68311
>  kbdmux_kbd_intr_timo                                          68311
>  lboltcb                                                       68311
>  mii_phy_update                                                68311
>  mii_tick                                                      68311
>  nd6_timer                                                     68311
>  rlphy_service                                                 68311
>  rlphy_status                                                  68311
>  sis_tick                                                      68311
>  ntp_update_second                                             68847
>  ttyoutput                                                     69581
>  kern_select                                                   69784
>  select                                                        69784
>  ffs_isfreeblock                                               73461
>  ffs_setblock                                                  73461
>  clear_selinfo_list                                            75128
>  g_slice_start                                                 76942
>  g_std_done                                                    76942
>  fork                                                          78596
>  sigonstack                                                    80762
>  fchdir                                                        81722
>  ffs_mapsearch                                                 83612
>  pargs_free                                                    84757
>  pargs_alloc                                                   84767
>  ffs_alloccgblk                                                88227
>  ffs_clrblock                                                  88227
>  pmap_ts_referenced                                            92202
>  ufs_readdir                                                   96142
>  vnode_pager_dealloc                                           97268
>  rijndaelKeySetupEnc                                           98242
>  ufs_bmaparray                                                 98697
>  ffs_alloc                                                    107477
>  mb_dupcl                                                     119087
>  tcp_output                                                   121581
>  sleepq_remove                                                121724
>  rijndaelEncrypt                                              122990
>  ffs_alloccg                                                  125682
>  ffs_statfs                                                   127909
>  kern_readlink                                                128586
>  readlink                                                     128586
>  putc                                                         130391
>  elf32_freebsd_fixup                                          131802
>  elf32_get_brandinfo                                          131802
>  exec_copyout_strings                                         131802
>  exec_new_vmspace                                             131802
>  exec_setregs                                                 131802
>  execsigs                                                     131802
>  sigacts_shared                                               131802
>  childproc_exited                                             132476
>  sigparent                                                    132476
>  sys_exit                                                     132485
>  acct_process                                                 133093
>  cpu_exit                                                     133093
>  exit1                                                        133093
>  ktr_drain                                                    133093
>  ktrace_assert                                                133093
>  ktrace_enter                                                 133093
>  ktrace_exit                                                  133093
>  ktrprocexit                                                  133093
>  leavepgrp                                                    133093
>  pfs_exit                                                     133093
>  proc_dtor                                                    133093
>  sched_exit                                                   133093
>  semexit_myhook                                               133093
>  sigacts_free                                                 133093
>  cpu_fork                                                     133103
>  fork1                                                        133103
>  lim_hold                                                     133103
>  microuptime                                                  133103
>  proc_ctor                                                    133103
>  pstats_fork                                                  133103
>  sched_fork                                                   133103
>  sigacts_alloc                                                133103
>  sigacts_copy                                                 133103
>  uifind                                                       133215
>  exec_elf32_imgact                                            133429
>  funsetownlst                                                 133487
>  uilookup                                                     133561
>  sched_exit_ksegrp                                            133980
>  sched_fork_ksegrp                                            133991
>  pargs_hold                                                   134069
>  selscan                                                      134878
>  do_tdsignal                                                  135385
>  sigtd                                                        135385
>  tdsignal                                                     135385
>  frag6_slowtimo                                               136619
>  igmp_slowtimo                                                136619
>  ip_slowtimo                                                  136619
>  pfslowtimo                                                   136619
>  tcp_slowtimo                                                 136619
>  tcp_timer_2msl_tw                                            136619
>  fork_return                                                  136807
>  cpu_thread_exit                                              137404
>  sched_exit_thread                                            137404
>  sched_thread_exit                                            137404
>  sched_fork_thread                                            137415
>  fork_exit                                                    137416
>  callout_init                                                 137538
>  mb_ctor_clust                                                139714
>  mb_dtor_clust                                                139714
>  sched_newthread                                              142693
>  lim_free                                                     146337
>  cpu_thread_clean                                             147169
>  VOP_READDIR_APV                                              147263
>  getdirentries                                                147263
>  sysarch                                                      149505
>  vfs_bio_clrbuf                                               150883
>  pmap_remove_page                                             151732
>  ffs_blkfree                                                  152259
>  fstatfs                                                      154346
>  kern_fstatfs                                                 154346
>  soo_poll                                                     155414
>  sopoll                                                       155414
>  ffs_fragacct                                                 157596
>  issetugid                                                    161953
>  ffs_blkpref_ufs2                                             162168
>  bucket_alloc                                                 162225
>  ttstart                                                      162521
>  ptsstart                                                     162557
>  ptcwakeup                                                    163661
>  b_to_q                                                       165433
>  mprotect                                                     168855
>  gettimeofday                                                 172532
>  microtime                                                    172532
>  elf32_check_header                                           173688
>  exec_check_permissions                                       173688
>  exec_map_first_page                                          173688
>  exec_unmap_first_page                                        173688
>  getrlimit                                                    175760
>  ffs_clusteracct                                              175782
>  ffs_hashalloc                                                176207
>  knlist_cleardel                                              177515
>  ffs_isblock                                                  177812
>  knlist_destroy                                               178404
>  knlist_init                                                  178426
>  pmap_activate                                                186384
>  ffs_bufwrite                                                 186437
>  kern_wait                                                    186914
>  wait4                                                        186914
>  vn_access                                                    186916
>  pmap_page_protect                                            191039
>  fpstate_drop                                                 191981
>  kthread_suspend_check                                        204919
>  pmap_remove_pages                                            210388
>  pargs_drop                                                   210741
>  sigqueue_delete_proc                                         219248
>  sigqueue_delete_set_proc                                     219248
>  uma_zfree_internal                                           219873
>  sigqueue_init                                                223686
>  selrecord                                                    225856
>  q_to_b                                                       226351
>  vhold                                                        226569
>  ufs_strategy                                                 229971
>  VOP_STRATEGY_APV                                             237103
>  bufstrategy                                                  237103
>  vop_strategy_pre                                             237103
>  zone_timeout                                                 246672
>  do_execve                                                    248797
>  exec_copyin_args                                             248797
>  exec_free_args                                               248797
>  execve                                                       248797
>  kern_execve                                                  248797
>  kmem_alloc_wait                                              248797
>  kmem_free_wakeup                                             248797
>  ptcpoll                                                      260640
>  stopprofclock                                                264895
>  eventhandler_prune_list                                      264998
>  ruadd                                                        266186
>  sigqueue_take                                                266186
>  chgproccnt                                                   266442
>  devfs_poll_f                                                 268251
>  giant_poll                                                   268251
>  munmap                                                       277920
>  uma_find_refcnt                                              279428
>  ip_reass                                                     286568
>  m_cat                                                        286568
>  pipe_write                                                   288398
>  pmap_clear_reference                                         289582
>  p_canwait                                                    298751
>  prison_check                                                 300270
>  vfs_vmio_release                                             302600
>  pipe_read                                                    307527
>  brelvp                                                       308094
>  bgetvp                                                       310934
>  getnewbuf                                                    310934
>  obreak                                                       311697
>  kse_release                                                  314648
>  tvtohz                                                       321724
>  scanc                                                        324279
>  ufsdirhash_adjfree                                           324493
>  logtimeout                                                   340436
>  icmp6_fasttimo                                               340437
>  igmp_fasttimo                                                340437
>  pffasttimo                                                   340437
>  miibus_readreg                                               341555
>  sis_mii_readreg                                              341555
>  sis_mii_sync                                                 341555
>  sis_miibus_readreg                                           341555
>  ufs_getlbns                                                  342152
>  elf32_load_section                                           344122
>  sf_buf_alloc                                                 345755
>  sf_buf_free                                                  345755
>  vnode_pager_alloc                                            346963
>  bufwait                                                      347083
>  bwait                                                        349014
>  vfs_mark_atime                                               359520
>  sleepq_timedwait_sig                                         360309
>  uma_slab_alloc                                               365368
>  bufwrite                                                     366343
>  ufsdirhash_lookup                                            380445
>  ufs_setattr                                                  380752
>  VOP_SETATTR_APV                                              380957
>  vop_setattr_post                                             380957
>  bdone                                                        386914
>  allocbuf                                                     393509
>  eventhandler_find_list                                       398108
>  getvnode                                                     400475
>  sysctl_handle_int                                            406974
>  fcntl                                                        408029
>  kern_fcntl                                                   408029
>  ffs_geom_strategy                                            429647
>  g_alloc_bio                                                  429647
>  g_disk_start                                                 429647
>  g_vfs_done                                                   429647
>  g_vfs_strategy                                               429647
>  ad_done                                                      433746
>  ad_strategy                                                  433746
>  bio_taskqueue                                                433746
>  callout_init_mtx                                             433746
>  devstat_start_transaction_bio                                433746
>  g_disk_done                                                  433746
>  sigqueue_collect_set                                         439385
>  sigqueue_move_set                                            439385
>  sysctl_old_user                                              447367
>  sysctl_find_oid                                              448015
>  sysctl_root                                                  448015
>  userland_sysctl                                              448015
>  useracc                                                      451635
>  reassignbuf                                                  454676
>  slot_fill                                                    465203
>  elf32_map_insert                                             476071
>  pmap_enter_object                                            486221
>  sigqueue_flush                                               489745
>  sleepq_wait_sig                                              491093
>  vnlru_free                                                   494481
>  g_io_check                                                   506589
>  g_io_deliver                                                 506589
>  g_io_request                                                 506589
>  g_clone_bio                                                  510688
>  vfs_busy_pages                                               513922
>  vtryrecycle                                                  515463
>  ast                                                          520359
>  incore                                                       523016
>  pmap_is_modified                                             523201
>  SHA256_Final                                                 526724
>  SHA256_Init                                                  526724
>  yarrow_hash_finish                                           526724
>  yarrow_hash_init                                             526724
>  access                                                       536834
>  crdup                                                        536834
>  kern_access                                                  536834
>  crcopy                                                       540045
>  crshared                                                     540045
>  bufdone_finish                                               546036
>  bufdone                                                      557655
>  runningbufwakeup                                             557655
>  sig_ffs                                                      564886
>  ffs_ifree                                                    583405
>  ufs_reclaim                                                  583405
>  pmap_ptelist_free                                            583944
>  pmap_ptelist_alloc                                           583968
>  bundirty                                                     600993
>  ffs_load_inode                                               610352
>  init_va_filerev                                              610352
>  ufs_vinit                                                    610352
>  vfs_hash_remove                                              612660
>  VOP_RECLAIM_APV                                              612690
>  delmntque                                                    612690
>  lockdestroy                                                  612690
>  vdestroy                                                     612690
>  vgonel                                                       612690
>  getbinuptime                                                 614655
>  cache_purge                                                  615638
>  vfs_hash_insert                                              641928
>  getnewvnode                                                  641956
>  insmntque                                                    641956
>  lockinit                                                     641956
>  vnode_destroy_vobject                                        659922
>  groupmember                                                  667819
>  crget                                                        672243
>  g_run_events                                                 678642
>  one_event                                                    678642
>  atkbd_timeout                                                679167
>  forward_roundrobin                                           679167
>  roundrobin                                                   679167
>  atkbd_check_char                                             679219
>  kbdc_data_ready                                              679219
>  ufsdirhash_hash                                              705032
>  vinvalbuf                                                    716363
>  vdrop                                                        716705
>  vnode_pager_setsize                                          724988
>  kern_lstat                                                   725025
>  lstat                                                        725025
>  cluster_read                                                 743292
>  cache_zap                                                    754792
>  buf_vlist_remove                                             762770
>  buf_vlist_add                                                765610
>  pmap_protect                                                 783797
>  cache_enter                                                  787915
>  ufs_close                                                    820555
>  pmap_copy                                                    832023
>  sleepq_wait                                                  835077
>  lseek                                                        835793
>  vn_finished_secondary_write                                  837230
>  vn_start_secondary_write                                     837230
>  sleepq_check_signals                                         851391
>  sleepq_catch_signals                                         851402
>  pmap_invalidate_all                                          855955
>  smp_invltlb                                                  855955
>  device_get_ivars                                             867492
>  pci_cfgdisable                                               867492
>  pci_cfgenable                                                867492
>  pci_cfgregread                                               867492
>  pci_read_config_method                                       867492
>  pcireg_cfgread                                               867492
>  ufs_lookup                                                   886150
>  ufsdirhash_build                                             886150
>  VOP_CACHEDLOOKUP_APV                                         886162
>  pmap_qremove                                                 915945
>  biodone                                                      940335
>  devstat_end_transaction                                      940335
>  devstat_end_transaction_bio                                  940335
>  devstat_start_transaction                                    940335
>  g_destroy_bio                                                940335
>  cursig                                                       963325
>  pmap_qenter                                                  973069
>  ufs_open                                                     994243
>  vn_write                                                     997191
>  ffs_write                                                    999072
>  ffs_update                                                  1009479
>  g_trace                                                     1013178
>  VOP_WRITE_APV                                               1013328
>  ffs_blkatoff                                                1016818
>  clock_gettime                                               1020062
>  kern_clock_gettime                                          1020062
>  ffs_balloc_ufs2                                             1061860
>  default_pager_dealloc                                       1201063
>  sigprocmask                                                 1203982
>  uifree                                                      1213768
>  uihold                                                      1213792
>  cv_broadcastpri                                             1233370
>  devfs_ioctl_f                                               1241467
>  ioctl                                                       1244363
>  ffs_read                                                    1316640
>  atkbd_lock                                                  1358334
>  kbdc_lock                                                   1358334
>  pmap_addr_hint                                              1361185
>  sis_mii_send                                                1366220
>  write                                                       1409154
>  kern_writev                                                 1410810
>  dofilewrite                                                 1417624
>  fget_write                                                  1417624
>  flushbuflist                                                1434143
>  kern_stat                                                   1484422
>  stat                                                        1484422
>  mmap                                                        1485435
>  lim_max                                                     1492987
>  pmap_copy_page                                              1534022
>  devfs_fp_check                                              1644296
>  dev_relthread                                               1650776
>  dev_refthread                                               1650778
>  and_region                                                  1692803
>  scrn_timer                                                  1692803
>  scrn_update                                                 1692803
>  vn_statfile                                                 1774785
>  fpu_clean_state                                             1791331
>  fpurstor                                                    1791331
>  fstat                                                       1814513
>  kern_fstat                                                  1814513
>  vn_closefile                                                1831175
>  VOP_CLOSE_APV                                               1831179
>  vn_close                                                    1831179
>  npxsave                                                     1864004
>  fpusave                                                     1864964
>  npxdna                                                      1871838
>  buf_dirty_count_severe                                      1876264
>  pmap_invalidate_range                                       1889014
>  smp_invlpg_range                                            1889014
>  pmap_remove_pte                                             1902880
>  vnode_create_vobject                                        2003180
>  VOP_OPEN_APV                                                2004892
>  default_pager_haspage                                       2014058
>  close                                                       2038540
>  knote_fdclose                                               2046418
>  vfs_timestamp                                               2103068
>  ufs_root                                                    2153919
>  bwillwrite                                                  2175502
>  resetpriority                                               2190250
>  resetpriority_thread                                        2207699
>  ufs_getattr                                                 2262255
>  dev2udev                                                    2262417
>  bdwrite                                                     2327914
>  vfs_clean_pages                                             2327914
>  bdirty                                                      2335044
>  closef                                                      2412933
>  timeout                                                     2444851
>  smp_invlpg                                                  2476656
>  pmap_invalidate_page                                        2482932
>  vn_read                                                     2523258
>  VOP_READ_APV                                                2623117
>  pmap_remove                                                 2669971
>  ufs_readlink                                                2809462
>  VOP_READLINK_APV                                            2810717
>  vfs_setdirty                                                2832887
>  g_bioq_unlock                                               2893848
>  kern_readv                                                  2896728
>  read                                                        2896728
>  ffs_vget                                                    2899723
>  dofileread                                                  2899930
>  fget_read                                                   2899930
>  kern_sigaction                                              2900402
>  sigaction                                                   2900402
>  vbusy                                                       3052468
>  vn_stat                                                     3111934
>  nanotime                                                    3183618
>  suser                                                       3266322
>  sleepq_timeout                                              3287385
>  bintime                                                     3356150
>  pmap_remove_entry                                           3392584
>  ufs_itimes                                                  3447738
>  brelse                                                      3454075
>  free_pv_entry                                               3496319
>  vn_finished_write                                           3506829
>  vn_start_write                                              3506829
>  pmap_enter_quick                                            3553775
>  vfree                                                       3576217
>  NDFREE                                                      3640468
>  VOP_LEASE_APV                                               3654788
>  vop_null                                                    3657466
>  SHA256_Transform                                            3686386
>  sleepq_timedwait                                            3716734
>  vn_isdisk                                                   4050322
>  sleepq_check_timeout                                        4077042
>  sleepq_set_timeout                                          4077043
>  fget                                                        4123679
>  vop_stdgetwritemount                                        4317168
>  VOP_GETWRITEMOUNT_APV                                       4330551
>  suser_cred                                                  4431260
>  bqrelse                                                     4461237
>  vfs_ref                                                     4471533
>  bread                                                       4844340
>  breada                                                      4844340
>  breadn                                                      4844340
>  g_bioq_lock                                                 4889062
>  g_bioq_first                                                4953323
>  sched_lend_prio                                             5081406
>  sched_rem                                                   5110444
>  adjustrunqueue                                              5113385
>  sched_unlend_prio                                           5173120
>  turnstile_broadcast                                         5173120
>  turnstile_setowner                                          5173120
>  turnstile_unpend                                            5173120
>  propagate_priority                                          5177099
>  turnstile_wait                                              5177298
>  msleep                                                      5332074
>  sched_sleep                                                 5402955
>  sleepq_resume_thread                                        5403202
>  sleepq_switch                                               5403209
>  sleepq_add                                                  5403213
>  sched_wakeup                                                5403831
>  setrunnable                                                 5404089
>  kern_open                                                   5471033
>  open                                                        5471033
>  vn_open                                                     5471042
>  vn_open_cred                                                5471042
>  buf_splay                                                   5472022
>  flsl                                                        5496868
>  falloc                                                      5515910
>  depart                                                      5767006
>  sosend_dgram                                                5781923
>  m_move_pkthdr                                               5781957
>  m_prepend                                                   5781957
>  ip_savecontrol                                              5781975
>  udp_append                                                  5781975
>  udp_output                                                  5781986
>  udp_send                                                    5781986
>  udp_input                                                   5782176
>  in_broadcast                                                5782204
>  sbappendaddr_locked                                         5783112
>  sowakeup                                                    5784068
>  m_dup_pkthdr                                                5802493
>  m_tag_copy_chain                                            5802493
>  swi_net                                                     5833970
>  netisr_processqueue                                         5833982
>  crhold                                                      5836879
>  m_copym                                                     5838871
>  in_delayed_cksum                                            5840687
>  ip_output                                                   5840696
>  rtalloc_ign                                                 5840734
>  in_pcblookup_hash                                           5840826
>  in_matroute                                                 5840884
>  rn_match                                                    5840884
>  rtalloc1                                                    5840884
>  arpresolve                                                  5876639
>  rt_check                                                    5876639
>  ether_output_frame                                          5876687
>  if_start                                                    5876687
>  sis_encap                                                   5876687
>  sis_start                                                   5876687
>  ether_output                                                5876689
>  sis_startl                                                  5877451
>  sis_txeof                                                   5878818
>  sis_rxeof                                                   5909426
>  jailed                                                      5917724
>  doselwakeup                                                 5947747
>  selwakeuppri                                                5947747
>  m_copyback                                                  6010554
>  ip_fastforward                                              6069576
>  ip_input                                                    6069582
>  ether_demux                                                 6069678
>  ether_input                                                 6069678
>  netisr_dispatch                                             6069678
>  sis_newbuf                                                  6069678
>  if_handoff                                                  6069684
>  legacy_setsoftnet                                           6069684
>  mb_ctor_pack                                                6087726
>  mb_dtor_pack                                                6087726
>  mtx_pool_alloc                                              6158212
>  pmap_remove_all                                             6166153
>  mb_free_ext                                                 6346527
>  device_get_parent                                           6462125
>  sis_intr                                                    6500277
>  random_harvest                                              6503437
>  random_harvest_internal                                     6503437
>  random_process_event                                        6503437
>  crfree                                                      6509084
>  lock_destroy                                                6692751
>  mtx_destroy                                                 6692751
>  mtx_init                                                    6754042
>  enroll                                                      6754087
>  lock_init                                                   6754087
>  tcp_isn_tick                                                6754846
>  getmicrotime                                                6754847
>  apic_idt_to_irq                                             6934057
>  lapic_handle_intr                                           6934057
>  knote                                                       7069057
>  dev_lock                                                    7357232
>  dev_unlock                                                  7357232
>  knlist_mtx_locked                                           7424206
>  getmicrouptime                                              7514614
>  bremfree                                                    7604378
>  getblk                                                      7758075
>  uiomove                                                     7762519
>  bremfreel                                                   7915312
>  malloc_type_freed                                           8464590
>  free                                                        8464596
>  malloc                                                      8466037
>  malloc_type_zone_allocated                                  8466037
>  mtrash_ctor                                                 8467002
>  mtrash_dtor                                                 8467554
>  gbincore                                                    8681360
>  namei                                                       8743299
>  lim_cur                                                     9143240
>  sched_clock                                                 9156890
>  statclock                                                   9156890
>  wakeup                                                      9323639
>  sleepq_broadcast                                            9323654
>  rman_get_bushandle                                          9524741
>  rman_get_bustag                                             9524741
>  vfs_page_set_valid                                          9547431
>  turnstile_lookup                                           10350418
>  turnstile_lock                                             10350449
>  pmap_clear_modify                                          10425835
>  pmap_zero_page                                             10537624
>  softclock                                                  10563984
>  sched_prio                                                 10704461
>  lim_rlimit                                                 10811987
>  device_get_softc                                           10916024
>  ffsl                                                       11109418
>  vfs_busy                                                   11258156
>  vfs_unbusy                                                 11258156
>  vfs_rel                                                    11412521
>  lookup                                                     11547368
>  removechild                                                11549403
>  insertchild                                                11549446
>  itismychild                                                11549446
>  soreceive                                                  11566078
>  m_length                                                   11622986
>  signotify                                                  11755058
>  m_adj                                                      11859824
>  pfil_run_hooks                                             11910280
>  bus_dmamap_destroy                                         11946365
>  vfs_hash_get                                               12020164
>  vrele                                                      12674838
>  kern_sigprocmask                                           12767770
>  pmap_try_insert_pv_entry                                   13658091
>  sleepq_release                                             13967599
>  SHA256_Update                                              14827773
>  yarrow_hash_iterate                                        14827773
>  sched_priority                                             15785807
>  vfs_hash_index                                             16276696
>  in_cksum_skip                                              17558324
>  sleepq_lookup                                              18135521
>  pmap_zero_page_idle                                        18644727
>  sleepq_lock                                                19370808
>  vref                                                       20723084
>  v_incr_usecount                                            21365040
>  m_freem                                                    23287791
>  bus_dmamap_create                                          23598672
>  sis_dma_map_desc_ptr                                       23598672
>  m_tag_delete_chain                                         23617781
>  swi_sched                                                  23746415
>  bus_dmamap_load                                            24032418
>  callout_reset                                              28236622
>  syscall                                                    29564295
>  ufs_inactive                                               31776663
>  trap_pfault                                                32032566
>  unlock_and_deallocate                                      32032628
>  vnode_pager_lock                                           32032721
>  pmap_enter                                                 32044341
>  pmap_enter_quick_locked                                    35554955
>  lockcount                                                  38531431
>  cpu_idle                                                   41449135
>  mp_grab_cpu_hlt                                            41449135
>  vfs_cache_lookup                                           41996117
>  ufs_access                                                 43255091
>  vaccess                                                    43259913
>  maybe_preempt                                              44787946
>  pmap_allocpte                                              45162843
>  mb_ctor_mbuf                                               48403213
>  mb_dtor_mbuf                                               48403214
>  pmap_pte_quick                                             48771986
>  DELAY                                                      50300802
>  VOP_GETATTR_APV                                            54408640
>  trash_ctor                                                 54630691
>  trash_dtor                                                 54630870
>  VOP_INACTIVE_APV                                           55931633
>  vinactive                                                  55931633
>  v_decr_useonly                                             55931638
>  choosethread                                               60178319
>  sched_choose                                               60178319
>  runq_add                                                   60311840
>  runq_remove                                                60311840
>  maybe_resched                                              60820837
>  sched_userret                                              61139013
>  setrunqueue                                                62078633
>  pmap_is_prefaultable                                       66700133
>  v_decr_usecount                                            66802829
>  mi_switch                                                  66921319
>  sched_switch                                               66921319
>  pmap_insert_entry                                          67077849
>  rdtsc                                                      67081484
>  sched_add                                                  67188820
>  hardclock                                                  68848801
>  hardclock_cpu                                              68848801
>  tc_ticktock                                                68848801
>  tc_windup                                                  68848801
>  nanouptime                                                 69664848
>  ffs_lock                                                   70801892
>  binuptime                                                  75034771
>  vop_stdlock                                                76203985
>  pmap_unuse_pt                                              79242610
>  get_pv_entry                                               80735940
>  sched_runnable                                             82898270
>  uma_zfree_arg                                              88154753
>  uma_dbg_free                                               88156215
>  uma_zalloc_arg                                             88296677
>  uma_dbg_alloc                                              88661958
>  cache_lookup                                               93651049
>  VOP_LOOKUP_APV                                             93654135
>  vop_lookup_post                                            93654135
>  vop_lookup_pre                                             93654135
>  VOP_ACCESS_APV                                             95933070
>  v_upgrade_usecount                                        101369711
>  vget                                                      101369783
>  vput                                                      110059629
>  runq_choose                                               120356638
>  vdropl                                                    123825813
>  vholdl                                                    123854336
>  lapic_handle_timer                                        137697603
>  lapic_eoi                                                 144631660
>  VOP_UNLOCK_APV                                            145034791
>  vop_stdunlock                                             145034791
>  vop_unlock_post                                           145034791
>  vop_unlock_pre                                            145034791
>  VOP_LOCK_APV                                              147005877
>  vop_lock_post                                             147005877
>  vop_lock_pre                                              147005877
>  vn_lock                                                   147085632
>  runq_check                                                165796540
>  VOP_ISLOCKED_APV                                          166179847
>  lockstatus                                                166179847
>  vop_stdislocked                                           166179847
>  uma_dbg_getslab                                           167986756
>  acquire                                                   314290116
>  lockmgr                                                   331095072
>  critical_enter                                           4364385478
>  critical_exit                                            4364385478
>
> --
> John Birrell
> _______________________________________________
> freebsd-current_at_freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"
>