Re: RFC: New event timers infrastructure

From: Brandon Gooch <jamesbrandongooch_at_gmail.com>
Date: Sun, 20 Jun 2010 09:46:12 -0500
On Sun, Jun 20, 2010 at 1:17 AM, Alexander Motin <mav_at_freebsd.org> wrote:
> Brandon Gooch wrote:
>> I've been testing these patches since the first iteration
>> (et.20100606), and I haven't discovered any related issues.
>
> Thank you!
>
>> I am unclear about the number of interrupts I should expect from the
>> hpet0 device (compared to the 99 from the rtc at 100Hz), so here is
>> the output of vmstat -i with and without the "et" patches:
>>
>> With "et" patches:
>>
>> interrupt                          total       rate
>> irq1: atkbd0                         369          3
>> irq9: acpi0                          961          8
>> irq12: psm0                         1002          9
>> irq18: uhci5                         140          1
>> irq19: uhci2 ehci0*                 4823         45
>> irq20: hpet0                       23893        223
>> irq23: uhci3 ehci1                    11          0
>> irq256: vgapci0                     1031          9
>> irq257: hdac0                         14          0
>> irq258: iwn0                        4258         39
>> irq259: bge0                           1          0
>> Total                              36503        341
>>
>> Without "et" patches:
>>
>> interrupt                          total       rate
>> irq1: atkbd0                         449          2
>> irq0: clk                          17334         99
>> irq9: acpi0                         1701          9
>> irq12: psm0                         8784         50
>> irq18: uhci5                         188          1
>> irq19: uhci2 ehci0*                 5828         33
>> irq23: uhci3 ehci1                    11          0
>> irq256: vgapci0                     1896         10
>> irq257: hdac0                         14          0
>> irq258: iwn0                       29571        169
>> irq259: bge0                           1          0
>> Total                              65777        378
>>
>> And lastly, the values of the kern.eventtimer sysctls:
>>
>> $ sysctl kern.eventtimer
>> kern.eventtimer.choice: HPET(450) HPET1(440) HPET2(440) HPET3(440) i8254(100)
>> kern.eventtimer.timer2: HPET1
>> kern.eventtimer.timer1: HPET
>> kern.eventtimer.singlemul: 4
>
> I don't see there neither LAPIC, nor RTC timer. I suppose you have
> disabled them via device hints. I also suppose you've done it to left
> with only 100Hz IRQs of i8254 timer. Now, with the patch, these two are
> still disabled, but system got 4 more HPET timers. As they have higher
> precedence, two of them were taken (HPET and HPET1). Number if
> interrupts can be explained by the line:
>
> Starting kernel event timers: HPET _at_ 100Hz, HPET1 _at_ 128Hz

Yes, I had the LAPIC and RTC timers disabled via my /boot/loader.conf,
as per your suggestions here:

http://wiki.freebsd.org/TuningPowerConsumption

> As result, you've got 228Hz IRQs (which then redistributed to every
> CPU). As your HPET timer doesn't support FSB interrupts, all it's timers
> share same IRQ vector, seen as "hpet0".

Understood.

> If you wish to get back to 100Hz IRQs as before, you may remove your
> previous clock-disabling hints and add instead:
>
> kern.eventtimer.timer1=HPET
> kern.eventtimer.timer2=NONE
> kern.eventtimer.singlemul=1

In by /boot/loader.conf, I now have:

# Power Saving
kern.hz="100"
#hint.apic.0.clock="0"
#hint.atrtc.0.clock="0"
hint.p4tcc.0.disabled="1"
hint.p4tcc.1.disabled="1"
hint.acpi_throttle.0.disabled="1"
hint.acpi_throttle.1.disabled="1"
hw.pci.do_power_nodriver="3"

In /etc/sysctl.conf, I have as suggested:

kern.eventtimer.timer1=HPET
kern.eventtimer.timer2=NONE
kern.eventtimer.singlemul=1

In my /etc/rc.conf, I'm still using:

performance_cpu_freq="NONE"
economy_cpu_freq="NONE"
performance_cx_lowest="C3"
economy_cx_lowest="C3"

I'm running powerd:

powerd_enable="YES"
powerd_flags="-a adaptive -b adaptive -n adaptive"

But in my dmesg, I see:

# dmesg | grep Starting
Starting kernel event timers: LAPIC _at_ 100Hz, HPET _at_ 128Hz
Starting kernel event timers: LAPIC _at_ 400Hz, NONE _at_ 0Hz
Starting kernel event timers: HPET _at_ 200Hz, NONE _at_ 0Hz

So it seems as if the HPET doesn't honor kern.hz setting? Maybe I
still need to explicitly disable the apic timers...

Would you suggest using LAPIC as opposed to HPET? I have seen power
savings being able to use C3 state, but if tickless kernel eventually
buys those savings back, perhaps LAPIC would be better...

> As result, you will have single timer, running at HZ rate. Instead of
> HPET there you may choose any timer:
>  LAPIC - it is per-CPU, so saves on IPI interrupts, supports one-shot
> mode and so suitable for further tickless kernel, but it doesn't work in
> C3 state;

So if LAPIC is disabled in C3 state, is there a possibility that the
"on-the-fly" method of changing clocks could kick in when the CPU
enters C3 state, switching to the HPET when it happens?

>  HPET{x} - on this hardware it can't be used as per-CPU, it supports
> one-shot mode, but less suitable for further tickless kernel, as CPUs
> can't run independently;

Since the HPET supports running while in C3, is there a strategy that
could be used to provide a tickless mode with both LAPIC and HPET
being used dynamically and interchangebly? I feel as though the
questions I'm trying to ask don't make much sense, my apologies :)

>  i8254 - somewhat faster, as it doesn't needs status reading, but due to
> being also a timecounter it can't be used in one-shot mode;
>  RTC - somewhat slower, has limited set of supported frequencies (powers
> of 2), only periodic mode.

Anyway, here is a glimpse into the VM interrupts stats using current
(stated) configuration:

$ vmstat -i
interrupt                          total       rate
irq1: atkbd0                        3968          2
irq9: acpi0                        11079          7
irq12: psm0                        64878         45
irq18: uhci5                        1131          0
irq19: uhci2 ehci0*                 7149          5
irq20: hpet0                      281935        197
irq23: uhci3 ehci1                    11          0
cpu0:timer                          4417          3
irq256: vgapci0                    31240         21
irq257: hdac0                         14          0
irq258: iwn0                       39537         27
irq259: bge0                           1          0
cpu1:timer                          4269          2
Total                             449629        315

-Brandon
Received on Sun Jun 20 2010 - 12:46:13 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:04 UTC