Re: r343567 aka PAE vs non-PAE merge breaks i386 freebsd

On Thu, Feb 28, 2019 at 09:09:38AM -0800, John Baldwin wrote:
> On 2/23/19 8:39 AM, Steve Kargl wrote:
> > On Sat, Feb 23, 2019 at 08:32:23AM -0800, Conrad Meyer wrote:
> >> On Sat, Feb 23, 2019 at 12:44 AM Steve Kargl
> >> <sgk_at_troutmask.apl.washington.edu> wrote:
> >>> Ideas?
> >>> ...
> >>> +CPU: Intel(R) Core(TM)2 Duo CPU     T7250  _at_ 2.00GHz (1995.04-MHz 686-class CPU)
> >>>    Origin="GenuineIntel"  Id=0x6fd  Family=0x6  Model=0xf  Stepping=13
> >>
> >> https://ark.intel.com/content/www/us/en/ark/products/31728/intel-core-2-duo-processor-t7250-2m-cache-2-00-ghz-800-mhz-fsb.html
> >>
> >>> Intel® Virtualization Technology (VT-x) ‡  Yes
> >>> Intel® 64 ‡   Yes
> >>
> >>> Merom is the first Intel mobile processor to feature Intel 64 architecture.
> >>
> >> So, as a workaround, maybe run amd64?
> > 
> > This is the only i386 FreeBSD system that I have.  This
> > is the system where all the libm changes I've made have
> > been tested.  i386 floating point is different than 
> > amd64 floating point.  See npx.c and the history of any
> > of the long double functions that I've worked on.  If
> > this laptop does not run i386, there will be no testing
> > of libm changes on the architecture.
> 
> Yes, but we set the initial FPU control word for 32-bit binaries
> to match i386 when running i386 binaries under an amd64 kernel.
> 
> See these comments in sys/x86/include/fpu.h with which you are
> likely familiar:
> 
> /*
>  * The hardware default control word for i387's and later coprocessors is
>  * 0x37F, giving:
>  *
>  *	round to nearest
>  *	64-bit precision
>  *	all exceptions masked.
>  *
>  * FreeBSD/i386 uses 53 bit precision for things like fadd/fsub/fsqrt etc
>  * because of the difference between memory and fpu register stack arguments.
>  * If its using an intermediate fpu register, it has 80/64 bits to work
>  * with.  If it uses memory, it has 64/53 bits to work with.  However,
>  * gcc is aware of this and goes to a fair bit of trouble to make the
>  * best use of it.
>  *
>  * This is mostly academic for AMD64, because the ABI prefers the use
>  * SSE2 based math.  For FreeBSD/amd64, we go with the default settings.
>  */
> #define	__INITIAL_FPUCW__	0x037F
> #define	__INITIAL_FPUCW_I386__	0x127F
> #define	__INITIAL_NPXCW__	__INITIAL_FPUCW_I386__
> #define	__INITIAL_MXCSR__	0x1F80
> #define	__INITIAL_MXCSR_MASK__	0xFFBF
> 
> And this code in ia32_setregs() in sys/amd64/ia32/ia32_signal.c
> to set the initial register values for i386 processes:
> 
> /*
>  * Clear registers on exec
>  */
> void
> ia32_setregs(struct thread *td, struct image_params *imgp, u_long stack)
> {
> ...
> 	pcb->pcb_initial_fpucw = __INITIAL_FPUCW_I386__;
> ...
> }
> 
> This matches what exec_setregs() in sys/i386/i386/machdep.c does:
> 
> /*
>  * Reset registers to default values on exec.
>  */
> void
> exec_setregs(struct thread *td, struct image_params *imgp, u_long stack)
> {
> ...
> 	pcb->pcb_initial_npxcw = __INITIAL_NPXCW__;
> ...
> }
> 
> You can do all your tests directly on amd64 by just adding
> "-m32" to compile i386 binaries against the libraries in /usr/lib32
> and you will generate the same i386 binaries as if you were building
> on an i386 system.  If you are a bit more paranoid, you can install
> an i386 world and chroot into it and use that to test i386.  I do this
> myself (-m32) for testing i386 things.  I also run i386 VMs under
> bhyve on amd64 hosts.  I'm not sure your laptop's CPU can run i386
> VMs though, and you don't need a VM to test userland-only changes
> (I'm usually trying to test kernel changes).

Interesting.  Did not know I could use -m32 with any reliability.
Setting up my testing environment may be a challenge as I use
mpfr/gmp, so would need -m32 aware versions of those libraries.
I normally install whatever the port collection offers for mpfr/gmp.
I suppose I would need to install those independently to get
multilib support.  I would also need to compile 2 versions of my
testing framework (ie., additional support library).

The BIOS does have a enable/disable button for virtualization.
During the great drm-legacy-kmod event of the last month, enabling
virtualization locks up a i386 FreeBSD kernel very quickly.
Perhaps, virtualization works under amd64.  Guess I'll burn
an image onto a memstick an d give it a whirl.

> However, an amd64 kernel is going to be a more stable, better
> supported kernel for running i386 binaries than an i386 kernel
> at this point, and that will become even more true in the future.

This is interesting as well.  Does this mean that amd64 is now 
the only tier 1 platform and all other architectures are after
thoughts?

-- 
Steve