Re: The machdep.hyperthreading_allowed & ULE weirdness in 7.1

From: Robert Watson <rwatson_at_FreeBSD.org>
Date: Mon, 23 Feb 2009 10:08:37 +0000 (GMT)
On Sun, 22 Feb 2009, Maxim Sobolev wrote:

> Hi Jeff,
>
> I have a single-CPU system with P4 HTT-enabled processor (7.1-RELEASE-p3), 
> kernel compiled with SCHED_ULE.

This is because machdep.hlt_logical_cpus doesn't do what you think it does. 
It causes HTT cores to invoke the hlt instruction in their idle loop, causing 
them to sleep until they receive an interrupt.  For 4BSD, which uses a "pull" 
model (on the whole) to bring work from work queues, this means that CPUs will 
go to sleep and remain that way unless they're actively receiving interrupts. 
For ULE, which uses "push" as well as "pull", threads will constantly be being 
shed from too-busy CPUs to apparently idle ones under load.  The only reliable 
way to disable hyperthreading is to do so using your BIOS setting, or use 
loader.conf to disable probing of the pics on unwanted cores, which will cause 
the CPUs not to be enumerated and hence not used.  We don't support taking 
CPUs fully offline with any scheduler, although you can use the cpuset 
facility to prevent threads from running on specific cores.  You could imagine 
teaching ULE (and presumably 4BSD) about policies such as "Don't use HTT 
cores", or perhaps just cpuset about those policies.

Robert N M Watson
Computer Laboratory
University of Cambridge

>
> Copyright (c) 1992-2009 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
>        The Regents of the University of California. All rights reserved.
> FreeBSD is a registered trademark of The FreeBSD Foundation.
> FreeBSD 7.1-RELEASE-p3 #0: Fri Feb 20 09:53:32 UTC 2009
>    root_at_foo.com:/usr/obj/usr/src/sys/SSP-PRODUCTION7
> Timecounter "i8254" frequency 1193182 Hz quality 0
> CPU: Intel(R) Pentium(R) 4 CPU 3.00GHz (2992.51-MHz 686-class CPU)
>  Origin = "GenuineIntel"  Id = 0xf34  Stepping = 4
>
> Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
>  Features2=0x441d<SSE3,DTES64,MON,DS_CPL,CNXT-ID,xTPR>
>  Logical CPUs per core: 2
> real memory  = 3758030848 (3583 MB)
> avail memory = 3674083328 (3503 MB)
> ACPI APIC Table: <IntelR AWRDACPI>
> FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
> cpu0 (BSP): APIC ID:  0
> cpu1 (AP): APIC ID:  1
> ioapic0: Changing APIC ID to 2
> ioapic0 <Version 2.0> irqs 0-23 on motherboard
> ioapic1 <Version 2.0> irqs 24-47 on motherboard
>
> machdep.cpu_idle_hlt: 1
> machdep.hlt_cpus: 0
> machdep.hlt_logical_cpus: 0
> machdep.logical_cpus_mask: 2
>
> If I flip machdep.hyperthreading_allowed sysctl, I still can see processes 
> scheduled to both logical CPUs, yet, if I check IRQ status in the systat -vm, 
> the CPU1 is not getting any timer interrupts, while CPU0 gets twice the 
> normal amount. I wonder if something is broken - I would expect no processes 
> to be scheduled to the CPU1.
>
> machdep.hyperthreading_allowed=1:
>  top: http://sobomax.sippysoft.com/~sobomax/ScreenShot458.png
>  systat -vm: http://sobomax.sippysoft.com/~sobomax/ScreenShot459.png
>
> machdep.hyperthreading_allowed=0:
>  top: http://sobomax.sippysoft.com/~sobomax/ScreenShot460.png
>  systat -vm: http://sobomax.sippysoft.com/~sobomax/ScreenShot461.png
>
> Please let me know if any other debug information is needed.
>
> -Maxim
> _______________________________________________
> freebsd-current_at_freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"
>
Received on Mon Feb 23 2009 - 09:08:38 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:39:42 UTC