Re: Reboot during startup with recent head

On Thu, Nov 08, 2012 at 05:33:53PM +0100, Joel Dahl wrote:
> On 08-11-2012 16:33, Konstantin Belousov wrote:
> > On Thu, Nov 08, 2012 at 01:23:13PM +0100, Joel Dahl wrote:
> > > On 08-11-2012 12:55, Andriy Gapon wrote:
> > > > on 08/11/2012 10:45 Joel Dahl said the following:
> > > > > Hi,
> > > > > 
> > > > > I rebuilt head (r242605) in a VMware virtual machine a couple of days ago, and
> > > > > now it won't boot anymore. The loader menu appears, the countdown starts, and boom,
> > > > > reboot. I tried a fresh head (r242700) but the problem remains.
> > > > > 
> > > > > Going back to r242205 solves the problem, so this was introduced somewhere
> > > > > between r242205 and r242605.
> > > > > 
> > > > > I managed to record the entire thing with my phone before I had to leave for
> > > > > work today. You can see it here:
> > > > > 
> > > > >   http://mirror.vnode.se/upload/head-reboots.mov
> > > > > 
> > > > > Any ideas? Any specific revision I should try to back out?
> > > > > 
> > > > 
> > > > Rather wild guess - try to back out the recent SMEP commit.
> > > 
> > > Well, it was a good guess. :-) I backed out revision 242433, rebuilt the kernel,
> > > and now it boots again.
> > 
> > The following patch should allow you to boot. Please show me the verbose dmesg
> > of the boot with the patch applied.
> 
> With this patch the machine is back to a non-booting state again. :-(
> 
> Anything else I should try?
Yes, please try the patch below. If it boots, I need a verbose dmesg.

My theory is that hypervisor is not aware of the extended features, and
do not filter it out from CPUID output. But, despite CPU announcing the
features, hypervisor does not support corresponding bits toggle in %cr4.
The attempt of the guest to enable e.g. FSGSBASE results in #gf during
the early stage of the boot, resulting in reset.

I just disabled extensions which need %cr4 configuration, when run
in VM.

diff --git a/sys/amd64/amd64/identcpu.c b/sys/amd64/amd64/identcpu.c
index 465316a..c0ea3ee 100644
--- a/sys/amd64/amd64/identcpu.c
+++ b/sys/amd64/amd64/identcpu.c
_at__at_ -134,7 +134,7 _at__at_ static struct {
 void
 printcpuinfo(void)
 {
-	u_int regs[4], i;
+	u_int regs[4], i, cpu_stdext_disable;
 	char *brand;
 
 	cpu_class = amd64_cpus[cpu].cpu_class;
_at__at_ -385,15 +385,36 _at__at_ printcpuinfo(void)
 			}
 
 			if (cpu_stdext_feature != 0) {
-				printf("\n  Standard Extended Features=0x%b",
-				    cpu_stdext_feature,
+				static const char stdext_b[] =
 				       "\020"
 				       "\001GSFSBASE"
 				       "\002TSCADJ"
 				       "\010SMEP"
 				       "\012ENHMOVSB"
-				       "\013INVPCID"
-				       );
+				       "\013INVPCID";
+				printf("\n  Standard Extended Features=0x%b",
+				    cpu_stdext_feature, stdext_b);
+
+				/*
+				 * Some hypervisors fail to filter out
+				 * unsupported extended features.  For
+				 * now, disable the extensions,
+				 * activation of which requires
+				 * setting a bit in CR4, and which VM
+				 * monitors do not support.
+				 */
+				if (cpu_feature2 & CPUID2_HV) {
+					cpu_stdext_disable =
+					    CPUID_STDEXT_FSGSBASE |
+					    CPUID_STDEXT_SMEP;
+				}
+				TUNABLE_INT_FETCH("hw.cpu_stdext_disable",
+				    &cpu_stdext_disable);
+				cpu_stdext_feature &= ~cpu_stdext_disable;
+				if (cpu_stdext_disable != 0) {
+					printf(" (Not disabled=0x%b)",
+					    cpu_stdext_feature, stdext_b);
+				}
 			}
 
 			if (via_feature_rng != 0 || via_feature_xcrypt != 0)