Re: STI, HLT in acpi_cpu_idle_c1

From: John Baldwin <jhb_at_FreeBSD.org>
Date: Fri, 25 Jun 2004 11:43:05 -0400
On Thursday 24 June 2004 07:32 pm, Gerrit Nagelhout wrote:
> The system still locked up without hlt on idle.  It is
> curious how it's always CPU 1 that ends up in this state
> though...is there anything special about it?  Is it
> possible that there is some kind of interaction with
> the BIOS?  In this case, CPU1 is still executing instructions,
> but doesn't think it has any interrupts to handle. (it
> keeps going in and out of the mp_grab_cpu_hlt function)
> I included the assembly of where the various
> cpus currently are below, as well as another APIC dump.
> CPU1 has interrupts enabled (as before) (EFLAGS=0x246).
> Does anyone have any recommendations for code that I could
> add to the idle loop to detect and recover from this
> condition?  It should be possible to read the icr_lo
> register, and make sure no interrupts are pending, and take
> some action based on this.
>
>
> c06afded <cpu_idle>:
> c06afded:       55                      push   %ebp
> c06afdee:       89 e5                   mov    %esp,%ebp
> c06afdf0:       e8 08 3b 00 00          call   c06b38fd <mp_grab_cpu_hlt>
> c06afdf5:       85 c0                   test   %eax,%eax
> c06afdf7:       75 1d                   jne    c06afe16 <cpu_idle+0x29>
> c06afdf9:       83 3d 14 f0 74 c0 00    cmpl   $0x0,0xc074f014
> c06afe00:(CPU0) 3e 74 13                je,pt  c06afe16 <cpu_idle+0x29>
> c06afe03:       fa                      cli
>
>
> c06b38fd <mp_grab_cpu_hlt>:
> c06b38fd:       55                      push   %ebp
> c06b38fe:       89 e5                   mov    %esp,%ebp
> c06b3900:       64 8b 15 28 00 00 00    mov    %fs:0x28,%edx
> c06b3907:       89 d1                   mov    %edx,%ecx
> c06b3909:       23 0d 64 8a 79 c0       and    0xc0798a64,%ecx
> c06b390f:(CPU1) 74 0e                   je     c06b391f
> <mp_grab_cpu_hlt+0x22>
> c06b3911:       fb                      sti
> c06b3912:       f4                      hlt
>
> c06b31d7 <smp_tlb_shootdown>:
> c06b31d7:       55                      push   %ebp
> c06b31d8:       89 e5                   mov    %esp,%ebp
> c06b31da:       83 ec 08                sub    $0x8,%esp
> c06b31dd:       89 5d fc                mov    %ebx,0xfffffffc(%ebp)
> c06b31e0:       8b 1d a0 99 76 c0       mov    0xc07699a0,%ebx
> c06b31e6:       83 eb 01                sub    $0x1,%ebx
> c06b31e9:       3e 74 3a                je,pt  c06b3226
> <smp_tlb_shootdown+0x4f>
> c06b31ec:       8b 45 0c                mov    0xc(%ebp),%eax
> c06b31ef:       a3 70 89 79 c0          mov    %eax,0xc0798970
> c06b31f4:       8b 45 10                mov    0x10(%ebp),%eax
> c06b31f7:       a3 74 89 79 c0          mov    %eax,0xc0798974
> c06b31fc:       b8 00 00 00 00          mov    $0x0,%eax
> c06b3201:       87 05 78 89 79 c0       xchg   %eax,0xc0798978
> c06b3207:       8b 45 08                mov    0x8(%ebp),%eax
> c06b320a:       89 04 24                mov    %eax,(%esp,1)
> c06b320d:       e8 c2 03 00 00          call   c06b35d4 <ipi_all_but_self>
> c06b3212:       a1 78 89 79 c0          mov    0xc0798978,%eax
> c06b3217:       39 d8                   cmp    %ebx,%eax
> c06b3219:       73 0b                   jae    c06b3226
> <smp_tlb_shootdown+0x4f>
> c06b321b:       f3 90                   repz nop
> c06b321d:(CPU2) a1 78 89 79 c0          mov    0xc0798978,%eax
> c06b3222:       39 d8                   cmp    %ebx,%eax
> c06b3224:       72 f5                   jb     c06b321b
> <smp_tlb_shootdown+0x44>
> c06b3226:       8b 5d fc                mov    0xfffffffc(%ebp),%ebx
> c06b3229:       89 ec                   mov    %ebp,%esp
> c06b322b:       5d                      pop    %ebp
> c06b322c:       c3                      ret
>
> c0556d12 <sched_runnable>:
> c0556d12:       55                      push   %ebp
> c0556d13:       89 e5                   mov    %esp,%ebp
> c0556d15:       83 ec 18                sub    $0x18,%esp
> c0556d18:       89 5d f8                mov    %ebx,0xfffffff8(%ebp)
> c0556d1b:       89 75 fc                mov    %esi,0xfffffffc(%ebp)
> c0556d1e:       be 01 00 00 00          mov    $0x1,%esi
> c0556d23:(CPU3) 64 a1 24 00 00 00       mov    %fs:0x24,%eax
> c0556d29:       69 c0 90 06 00 00       imul   $0x690,%eax,%eax
> c0556d2f:       8d 98 80 0c 76 c0       lea    0xc0760c80(%eax),%ebx
> c0556d35:       83 bb 8c 06 00 00 00    cmpl   $0x0,0x68c(%ebx)
> c0556d3c:       3e 0f 84 97 00 00 00    je,pt  c0556dda
> <sched_runnable+0xc8>
> c0556d43:       e8 91 09 ff ff          call   c05476d9 <critical_enter>
> c0556d48:       b8 04 00 00 00          mov    $0x4,%eax
>
>
> P3>dumpAllLocalApic
> CPU 0
> ID:    0x6000000
> TPR:   0x0
> PPR:   0x0
> icr_lo:0xf3
> APR:   0x0
> ISR0:  0x0
> ISR1:  0x0
> ISR2:  0x0
> ISR3:  0x0
> ISR4:  0x0
> ISR5:  0x0
> ISR6:  0x0
> ISR7:  0x0
> IRR0:  0x0
> IRR1:  0x0
> IRR2:  0x0
> IRR3:  0x0
> IRR4:  0x0
> IRR5:  0x0
> IRR6:  0x0
> IRR7:  0x0
> TMR0:  0x0
> TMR1:  0x0
> TMR2:  0x0
> TMR3:  0x0
> TMR4:  0x0
> TMR5:  0x0
> TMR6:  0x0
> TMR7:  0x0
> CPU 1
> ID:    0x7000000
> TPR:   0x0
> PPR:   0xf0
> icr_lo:0xf3
> APR:   0x0
> ISR0:  0x0
> ISR1:  0x0
> ISR2:  0x0
> ISR3:  0x0
> ISR4:  0x0
> ISR5:  0x0
> ISR6:  0x0
> ISR7:  0x8000000

This is why it isn't receiving interrupts.  It thinks one is still being 
serviced and is waiting on the EOI.   That is IPI_HARDCLOCK again.
Hmm, try this hack (you can turn idle_hlt back on if you want since it doesn't 
seem to have an effect):

Index: intr_machdep.c
===================================================================
RCS file: /usr/cvs/src/sys/i386/i386/intr_machdep.c,v
retrieving revision 1.6
diff -u -r1.6 intr_machdep.c
--- intr_machdep.c	28 May 2004 17:50:07 -0000	1.6
+++ intr_machdep.c	25 Jun 2004 15:42:00 -0000
_at__at_ -184,6 +184,10 _at__at_
 		clkintr_pending = 1;
 
 	if (ih != NULL && ih->ih_flags & IH_FAST) {
+#if 1
+		isrc->is_pic->pic_disable_source(isrc);
+		isrc->is_pic->pic_eoi_source(isrc);
+#endif
 		/*
 		 * Execute fast interrupt handlers directly.
 		 * To support clock handlers, if a handler registers
_at__at_ -202,7 +206,11 _at__at_
 			else
 				ih->ih_handler(ih->ih_argument);
 		}
+#if 0
 		isrc->is_pic->pic_eoi_source(isrc);
+#else
+		isrc->is_pic->pic_disable_source(isrc);
+#endif
 		error = 0;
 		critical_exit();
 	} else {

-- 
John Baldwin <jhb_at_FreeBSD.org>  <><  http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve"  =  http://www.FreeBSD.org
Received on Fri Jun 25 2004 - 13:42:36 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:37:59 UTC