Re: Boot loader stuck after first stage upgrading 11.2 to 12.0-RC2

>> 2018-11-29 18:43, Toomas Soome wrote:
>>> I just did push biosdisk updates to stable/12, I wonder if you could
>>> test those bits…

Myself wrote:
>> Thank you!  I haven't tried it yet, but I wonder whether this fix was
>> already incorporated into 12.0-RC3, which would make my rescue easier.
>> Otherwise I can build a stable/12 on another host and transplant
>> the problematic file(s) to the affected host - if I knew which files
>> to copy.

2018-12-02 18:59, Toomas wrote:
> The files are /boot/loader* binaries - to be exact, check which one is
> linked to /boot/loader. I can provide binaries if needed.
> [...]
> rgds,
> toomas

I got a maintenance window today so I tried with the new loader,
and it did not help.

More specifically:

As it comes with 12-RC2, the /boot/loader was hard linked with 
loader_lua.
Its size is 421888 bytes. So I concentrated on this loader.

I build a fresh stable/12 on another host, and copied the newly
built loader_lua (425984 bytes) to the /boot directory of the affected
host, deleted the file 'loader', and hard-linked loader_lua to loader.

The situation has not changed: the BTX loader lists all BIOS drives
C..J (disk0..disk7), then a spinner starts and gets stuck forever.
It never reaches the 'BIOS 635kB/3537856kB available memory' line.

While trying to restore the old /boot from 11.2, I tried booting
a live image from a 12.0-RC3 memory stick - and the loader got
stuck again, same as when booting from a disk.

So I had to boot from an 11.2 memstick to be able to regain control.

   Mark



>>>> On 29 Nov 2018, at 17:01, Mark Martinec 
>>>> <Mark.Martinec+freebsd_at_ijs.si> wrote:
>>>> After successfully upgraded three hosts from 11.2-p4 to 12.0-RC2 
>>>> (amd64,
>>>> zfs, bios), I tried my luck with one of our production hosts, and 
>>>> ended up
>>>> with a stuck loader after rebooting with a new kernel (after the 
>>>> first
>>>> stage of upgrade).
>>>> These were the steps, and all went smoothly and normally until a 
>>>> reboot:
>>>> freebsd-update upgrade -r 12.0-RC2
>>>> freebsd-update install
>>>> shutdown -r now
>>>> While booting, the 'BTX loader' comes up, lists the BIOS drives,
>>>> then the spinner below the list comes up and begins turning,
>>>> stuttering, and after a couple of seconds it grinds to a standstill
>>>> and nothing happens afterwards.
>>>> At this point the ZFS and the bootstrap loader is supposed to
>>>> come up, but it doesn't.
>>>> This host has too zfs pools, the system pool consists of two SSDs
>>>> in a zfs mirror (also holding a freebsd-boot partition each), the
>>>> other pool is a raidz2 with six JBOD disks on an LSI controller.
>>>> The gptzfsboot in both freebsd-boot partitions is fresh from 11.2,
>>>> both zpool versions are up-to-date with 11.2. The 'zpool status -v'
>>>> is happy with both pools.
>>>> After rebooting from an USB drive and reverting the /boot directory
>>>> to a previous version, the machine comes up normally again
>>>> with the 11.2-RELEASE-p4.
>>>> I found a file init.core in the / directory, slightly predating the
>>>> last reboot with a salvaged system - although it was probably not
>>>> a cause of the problem, but a consequence of the rescue operation.
>>>> It is unfortunate that this is a production host, so I can't play
>>>> much with it. One or two more quick experiments I can probably
>>>> afford, but not much more. Should I just first wait for the
>>>> official 12.0 release? Should I try booting with a 12.0 on USB
>>>> and try to import pools? Suggestions welcome.
>>>> Now that the /boot has been manually restored to the 11.2 state,
>>>> A SECOND QUESTION is about freebsd-update, which still thinks we are
>>>> in the middle of an upgrade procedure. Trying now to just update
>>>> the 11.2-RELEASE-p4 to 11.2-RELEASE-p5, the fetch complains:
>>>> # uname -a
>>>> FreeBSD xxx 11.2-RELEASE-p4 FreeBSD 11.2-RELEASE-p4
>>>> #
>>>> # freebsd-version
>>>> 11.2-RELEASE-p4
>>>> #
>>>> # freebsd-update fetch
>>>> src component not installed, skipped
>>>> You have a partially completed upgrade pending
>>>> Run '/usr/sbin/freebsd-update install' first.
>>>> Run '/usr/sbin/freebsd-update fetch -F' to proceed anyway.
>>>> So what is the right way to get rid of all traces of the
>>>> unsuccessful upgrade, and let freebsd-update believe we are cleanly
>>>> at 11.2-p4 ?  Removing /var/db/freebsd-update did not help.
>>>> Mark