Re: Recommended virtualization technique for debugging/developing FreeBSD

Peter Wemm wrote:
> On Tue, Feb 26, 2008 at 10:30 AM, Bill Moran <wmoran_at_potentialtech.com> wrote:
>> In response to Peter Schuller <peter.schuller_at_infidyne.com>:
>>
>>
>>  > Hello,
>>  >
>>  > I was wondering what people use, in the abscense of suitable actual hardware,
>>  > to debug/develop FreeBSD (the kernel in particular). I'm willing to resort to
>>  > almost any host, including Windows, as long as I have something reliable.
>>  >
>>  > I haven't had much luck with qemu (crashes), nor virtualbox (crashes). I was
>>  > going to go for vmware on Windows, but while it ran FreeBSD pretty well,
>>  > before I had even percolated the disk layout enough to trigger the bug
>>  > (required root-on-zfs) I was hoping to trigger, the vmware configuration tool
>>  > crapped out on me and produced a configuration it could not itself read.
>>  >
>>  > What do all you regular kernel developers use, if not physical hardware?
>>
>>  I know that bochs was used during some of the initial development of
>>  the amd64 port, because bochs can emulate amd64 on i386 hardware.
>>
>>  You're not going to see anything like impressive performance with bochs,
>>  but it will allow you to see _everything_ the kernel is doing, i.e., you
>>  can track each CPU instruction if you so desire.
> 
> For what its worth, I never used an emulator, nor even gdb, for
> bringing up FreeBSD/amd64 to multi-user.  It was all done with printf
> and a machine that booted up very very fast.  It was at loader prompt
> in about 4 seconds from a power cycle.
> 
> The initial kernels were netbooted with pxe and later disk booted on
> the machine that was set up for dual booting.  Not long after it made
> it to multi-user, things like libc etc were all statically linked and
> everything was seld compiled but with an external compiler.  I added
> most of the missing bits incrementally.  David made the toolchain in
> the tree work and eventually it self-compiled using the in-tree
> compiler and toolchain.
> 
> Most initial debugging was done with a super-low-level serial console,
> things like writing values to screen ram, 'hlt' intruction debugging
> to diagnose insta-reboots, etc.
> 
> The initial address space size was 1GB.   512M for user, 512M for
> kernel.  later came 4GB using i386's PAE strategies, then 512GB, then
> 128TB as I earned how to make pmap work.
> 
> It was a lot of fun, but I had no real debugging capabilities.  I
> thought about bochs but never used it.  gdb wasn't available either.
> I've never used serial gdb.
> 
> IA64 was different.  Doug Rabson used things like emulators/simulators
> like Simics extensively, which had remote debugger (gdb?) hooks and
> the like.
> 
> John Baldwin has been using qemu lately for bios/btx/loader work on x86.


I've been using qemu PXE booting an NFS image for some time now for file 
system development.  I use DHCP locally to give it the tftp info, and 
have a local tftpd that serves up a pxe booter to start up qemu.  Qemu 
boots and loads my custom built kernel(s), loads modules, etc.  I use 
the virtual serial port in qemu to do debugging, and have no virtual 
hard drives since it is completely NFS mounted.

My only complaint is the speed of qemu really, and its stability.  It's 
good enough, but vmware would be so much better.

Eric