Re: Periodical interrupt storm when playing game with USB keyboard

From: blubee blubeeme <gurenchan_at_gmail.com>
Date: Tue, 23 Jan 2018 20:06:24 +0800
On Tue, Jan 23, 2018 at 7:27 PM, Johannes Lundberg <johalun0_at_gmail.com>
wrote:

> Hi all
>
> Some quick dtracing with play causing lag, vs play not causing lag (that
> is not hold down any key on a usb keyboard for too long).
>
>
> # dtrace -n 'profile-997hz /arg0/ { _at_[func(arg0)]=count(); }'
>
> Lag version
> -- snip --
>   linuxkpi.ko`idr_find                                              7
>   kernel`nanotime                                                   8
>   kernel`__cap_rights_init                                          8
>   kernel`amd64_syscall                                              8
>   i915kms.ko`i915_gem_obj_lookup_or_create_vma                      8
>   kernel`selfdfree                                                  9
>   kernel`feed_matrix_S16LE                                         11
>   kernel`callout_lock                                              11
>   kernel`uma_zalloc_arg                                            11
>   kernel`z_feed_linear_S16LE                                       12
>   kernel`0xffffffff80f6877e                                        12
>   kernel`copyin                                                    12
>   kernel`tsc_get_timecount_low_lfence                              12
>   kernel`bzero                                                     13
>   kernel`fget_unlocked                                             14
>   i915kms.ko`gen8_ppgtt_insert_pte_entries                         14
>   kernel`callout_when                                              16
>   kernel`0xffffffff80f68fbc                                        26
>   kernel`kern_select                                               32
>   kernel`lock_mtx                                                  50
>   kernel`spinlock_enter                                            53
>   kernel`bcopy                                                     57
>   kernel`unlock_mtx                                               104
>   i915kms.ko`fw_domains_get                                       113
>
>
> *  kernel`ukbd_timeout                                             126
> kernel`callout_reset_sbt_on                                     128
> kernel`ukbd_interrupt                                           136*
> *  kernel`softclock_call_cc                                        192*
>   kernel`memcpy                                                   284
>   kernel`cpu_idle                                                4257
>
> *  kernel`spinlock_exit                                           4312
> kernel`lock_delay                                             11921*
>   kernel`acpi_cpu_idle                                          15265
>
A question on the mailing list about spinlocks and high cpu time:
https://lists.freebsd.org/pipermail/freebsd-questions/2008-October/183943.html

Although I think that for USB you can most likely fully replace the
spinlock with a lock free queue, stack allocator and a ring buffer.

Tony Van Eerd has been doing some very interesting work on lock free queue
and refined his implementation, his talk from this year's cpp con looks
very promising. You can check it out here:
https://www.youtube.com/watch?v=HP2InVqgBFM

Locks just have overhead that can't be avoided, with something like USB you
can use a heap allocator, a ring buffer and a lock free queue and that
should be able to get rid of the lock.

I remember looking at the USB stack a while ago, there's one global lock or
something like that and looking at the DTrace logs it's definitely that
causing issues somehwere.

It'll need testing though.

>
>
> No lag version
> -- snip --
>   kernel`free                                                      19
>   kernel`copyout                                                   20
>   kernel`copyin                                                    20
>   linuxkpi.ko`idr_find                                             21
>   kernel`selfdfree                                                 24
>   kernel`tsc_get_timecount_low_lfence                              25
>   kernel`__mtx_lock_flags                                          28
>   kernel`uma_zalloc_arg                                            30
>   kernel`bzero                                                     31
>   i915kms.ko`gen8_ppgtt_insert_entries                             31
>   drm.ko`drm_ioctl                                                 32
>   kernel`fget_unlocked                                             36
>   kernel`amd64_syscall                                             37
>   kernel`kern_select                                               49
>   i915kms.ko`gen8_ppgtt_insert_pte_entries                         61
>   kernel`0xffffffff80f68fbc                                        78
>   kernel`bcopy                                                     90
>   i915kms.ko`fw_domains_get                                       228
>   kernel`spinlock_exit                                            284
>   kernel`cpu_idle                                                4698
>   kernel`acpi_cpu_idle                                          36288
>
>
> I also tried rebuilding linux-c6_sdl-1.2 using get time functions in librt
> vs libc but no difference.
> Will dig deeper next time I have free time to spare.
>
>
>
> On Tue, Jan 23, 2018 at 2:33 AM, blubee blubeeme <gurenchan_at_gmail.com>
> wrote:
>
>>
>>
>> On Tue, Jan 23, 2018 at 9:48 AM, Adrian Chadd <adrian.chadd_at_gmail.com>
>> wrote:
>>
>>> Hi
>>>
>>> Yeah the timers eventually get coalesced unless someone's asking for a
>>> ridciulously accurate timer value.
>>>
>>> So is some driver asking for hyper-accurate callout timer that isn't
>>> being coalesced? hps, is there any useful debugging to try and find
>>> callouts that are requesting stupidly accurate timers? Maybe a dtrace
>>> probe on the callout schedule entry point?
>>>
>>>
>>>
>>> -adrian
>>> _______________________________________________
>>> freebsd-current_at_freebsd.org mailing list
>>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
>>> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_f
>>> reebsd.org"
>>>
>> I'd say dtrace might be able to get you in the right direction very
>> quickly.
>>
>> I used SDL in the past for android apps and the code is very Linux
>> specific. I am sure there's some Linux related timers in there somewhere
>> that FreeBSD is returning nothing from and that's what's killing the
>> performance.
>>
>> I can almost guarantee that none of the SDL designers and or programmers
>> use any *BSD systems.
>>
>> The easiest solution would be to go look at the timer code and implement
>> something that FreeBSD can work with and try to get that upstream.
>>
>> These are just a few of the issues that will crop up when devs try to
>> just use shims to hook into the Linux kernel. Do the design work up front
>> and implement things in a native way or enjoy the jank.
>>
>> DTrace should be able to point you in the right direction relatively
>> quickly.
>> The DTraceBook: https://www.amazon.com/DTrace-Dynamic-Tracing-
>> Solaris-FreeBSD/dp/0132091518
>>
>> Best
>>
>
>
Received on Tue Jan 23 2018 - 11:06:25 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:14 UTC