Re: LUA boot loader coming very soon

On Mon, Feb 12, 2018 at 10:12 AM, John Baldwin <jhb_at_freebsd.org> wrote:

> On Monday, February 12, 2018 08:27:27 AM Warner Losh wrote:
> > Greetings,
> >
> > As you may know, the Lua (http://www.lua.org) boot loader has been in
> the
> > works for some time. It started out life as a GSoC in 2014 by Pedro Souza
> > mentored by Wojciech A. Koszek. Rui Paulo created a svn project branch to
> > try to integrate it. I rebased that effort into a github branch which
> Pedro
> > Arthur fixed up. Over the past year, I've been cleaning up the boot
> loader
> > for other reasons, and found the time was ripe to start integrating this
> > into the tree. However, those integration efforts have taken a while as
> my
> > day-job work on the boot loader took priority. In the mean time, Ed Maste
> > and the FreeBSD Foundation funded Zakary Nafziger to enhance the original
> > GSoC Lua scripts to bring it closer to parity with the evolution of the
> > FORTH menu system since the GSoC project started.
> >
> > I'm pleased to announce that all these threads of development have
> > converged and I'll be pushing the FreeBSD Lua Loader later today. This
> > loader uses Lua as its scripting language instead of FORTH. While
> > co-existance is planned, the timeline for it is looking to be a few weeks
> > and I didn't want to delay pushing this into the tree for that.
> >
> > To try the loader, you'll need to build WITHOUT_FORTH=yes and
> > WITH_LOADER_LUA=yes. Fortunately, you needn't do a full world to do this,
> > you can do it in src/stand and install the result (be sure to have the
> > options for both the build and the install). This will replace your
> current
> > /boot/loader that is scripted with FORTH to one that's scripted with Lua.
> > It will install the lua scripts in /boot/lua. The boot is scripted with
> > /boot/lua/loader.lua instead of /boot/loader.rc. You are strongly advised
> > to create a backup copy of /boot/loader before testing (eg cp
> /boot/loader
> > /boot/loader_forth), since you'll need to boot that from boot2 if
> something
> > goes wrong. I've tested it extensively, though, with userboot.so and it's
> > test program, so all the initial kinks of finding the lua scripts, etc
> have
> > been worked out.
> >
> > While it's possible to build all the /boot/loader variants with Lua, I've
> > just tested a BIOS booting /boot/loader both with and without menus
> > enabled. I've not tested any of the other variants and the instructions
> for
> > testing some of them may be rather tedious (especially UEFI, if you want
> a
> > simple path to back out). Since there's not been full convergence
> testing,
> > you'll almost certainly find bumps in this system. Also, all the
> > build-system APIs are likely not yet final.
> >
> > I put  MFC after a month on the commit. Due to the heroic (dare I say
> > almost crazy) work of Kyle Evans on merging all the revs from -current to
> > 11, I'm planning a MFC to 11 after the co-existence issues are hammered
> > out. In 11, FORTH will be the default, and Lua will  be built by default,
> > but users will have to do something to use it. 12, both FORTH and Lua
> will
> > be built and installed, with Lua as default (barring unforeseen
> > complications). Once the co-existence stuff goes in, I imagine we'll make
> > the switch to Lua by default shortly after that. In 13, FORTH will be
> > removed unless there's a really really compelling case made to keep it.
> >
> > So please give it a spin and give me any feedback, documentation updates
> > and/or bug fixes. I'm especially interested in reviews from people that
> > have embedded Lua in other projects or experts in Lua that can improve
> the
> > robustness of the menu code.
>
> Do you have some memory usage numbers for LUA vs forth for the different
> BIOS loaders (text/data/bss sizes)?  For the EFI case we probably have lots
> of room, but for the non-EFI case we are constrained to 0xa0000 - 0xa000
> for the text/data/bss and stack (in some cases like PXE booting the top
> can be lower than 0xa0000).  I'm not sure if we have any other platforms
> with similar memory constraints.
>

Lua is about 60-70k larger than FORTH for this application:

Forth based:
-r-xr-xr-x  1 root  wheel  385024 Feb 12 14:14
/home/imp/roots/amd64/boot/loader
-r-xr-xr-x  1 root  wheel  450560 Feb 12 14:14
/home/imp/roots/amd64/boot/zfsloader
Lua based:
-r-xr-xr-x  1 root  wheel  450560 Feb 12 14:20
/home/imp/roots/amd64/boot/loader
-r-xr-xr-x  1 root  wheel  520192 Feb 12 14:20
/home/imp/roots/amd64/boot/zfsloader

So with Lua:
%  size loader.bin
    text    data     bss      dec       hex   filename
  411840   22264   48304   482408   0x75c68   loader.bin
% size zfsloader.bin
    text    data     bss      dec       hex   filename
  478712   22968   52368   554048   0x87440   zfsloader.bin

so we're at 555k out of 640k before we look at heap usage. Of course, these
numbers are only going to get a lot worse as we pile-in new crypto
features... I know that the normal loader works, but I've not tried the
zfsloader.

We might be able to do some slimming of LUA. I've honestly not tried
super-hard to do that, but do presently omit a lot of the standard
libraries which saves some space.

Warner