Re: [UEFI] Boot issues on some UEFI implementations

From: Warner Losh <imp_at_bsdimp.com>
Date: Fri, 27 Jul 2018 12:03:55 -0600
On Fri, Jul 27, 2018 at 11:05 AM, O. Hartmann <ohartmann_at_walstatt.org>
wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
>
> Am Fri, 27 Jul 2018 13:59:44 +0300
> Toomas Soome <tsoome_at_me.com> schrieb:
>
> > > On 27 Jul 2018, at 13:02, O. Hartmann <ohartmann_at_walstatt.org> wrote:
> > >
> > > On Fri, 27 Jul 2018 08:54:48 +0300
> > > Toomas Soome <tsoome_at_me.com> wrote:
> > >
> > >>> On 27 Jul 2018, at 08:46, O. Hartmann <ohartmann_at_walstatt.org>
> wrote:
> > >>>
> > >>> On Thu, 26 Jul 2018 19:23:43 +0300
> > >>> Toomas Soome <tsoome_at_me.com> wrote:
> > >>>
> > >>> (reply inline/at the end)
> > >>>
> > >>>
> > >>>>> On 26 Jul 2018, at 16:58, O. Hartmann <ohartmann_at_walstatt.org>
> wrote:
> > >>>>>
> > >>>>> On Wed, 25 Jul 2018 07:30:32 -0700 (PDT)
> > >>>>> "Rodney W. Grimes" <freebsd-rwg_at_pdx.rh.CN85.dnsmgr.net
> > >>>>> <mailto:freebsd-rwg_at_pdx.rh.CN85.dnsmgr.net>> wrote:
> > >>>>>>>> On 25 Jul 2018, at 12:10, O. Hartmann <o.hartmann_at_walstatt.org>
> wrote:
> > >>>>>>>>
> > >>>>>>>> On Wed, 25 Jul 2018 11:46:07 +0300
> > >>>>>>>> Toomas Soome <tsoome_at_me.com> wrote:
> > >>>>>>>>
> > >>>>>>>>>> On 25 Jul 2018, at 10:59, O. Hartmann <ohartmann_at_walstatt.org>
> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> On Tue, 24 Jul 2018 08:53:36 +0300
> > >>>>>>>>>> Toomas Soome <tsoome_at_me.com> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> Hello  Toomas Soome,
> > >>>>>>>>>>
> > >>>>>>>>>> I CC Allan Jude since I discovered something  weird today
> regarding
> > >>>>>>>>>> the UEFI boot capabilities of USB flash devices and SSDs. See
> below.
> > >>>>>>>>>>
> > >>>>>>>>>>>> On 24 Jul 2018, at 08:16, O. Hartmann <
> ohartmann_at_walstatt.org>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Mon, 23 Jul 2018 10:56:04 +0300
> > >>>>>>>>>>>> Toomas Soome <tsoome_at_me.com> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>>> On 23 Jul 2018, at 10:27, O. Hartmann <
> ohartmann_at_walstatt.org>
> > >>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Fri, 13 Jul 2018 18:44:23 +0300
> > >>>>>>>>>>>>>> Toomas Soome <tsoome_at_me.com <mailto:tsoome_at_me.com>>
> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> On 13 Jul 2018, at 17:44, O. Hartmann <
> o.hartmann_at_walstatt.org
> > >>>>>>>>>>>>>>>> <mailto:o.hartmann_at_walstatt.org>> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> -----BEGIN PGP SIGNED MESSAGE-----
> > >>>>>>>>>>>>>>>> Hash: SHA512
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Am Fri, 13 Jul 2018 14:26:51 +0300
> > >>>>>>>>>>>>>>>> Toomas Soome <tsoome_at_me.com <mailto:tsoome_at_me.com>
> > >>>>>>>>>>>>>>>> <mailto:tsoome_at_me.com <mailto:tsoome_at_me.com>>>
> > >>>>>>>>>>>>>>>> schrieb:
> > >>>>>>>>>>>>>>>>>> On 13 Jul 2018, at 14:00, O. Hartmann
> > >>>>>>>>>>>>>>>>>> <ohartmann_at_walstatt.org> wrote:
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> The problem is some kind of weird. I face UEFI boot
> problems
> > >>>>>>>>>>>>>>>>>> on GPT drives where the first partition begins at
> block 40
> > >>>>>>>>>>>>>>>>>> of the hdd/ssd.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> I have two host in private use based on an
> > >>>>>>>>>>>>>>>>>> outdated ASRock Z77-Pro4-M and Z77-Pro4 mainboard
> (IvyBridge,
> > >>>>>>>>>>>>>>>>>> Socket LGA1155). Both boards are equipted with the
> lates
> > >>>>>>>>>>>>>>>>>> official available AMI firmware revision, dating to
> 2013.
> > >>>>>>>>>>>>>>>>>> This is for the Z77-Pro4-M revision 2.0 (2013/7/23)
> and for
> > >>>>>>>>>>>>>>>>>> the Z77 Pro4 revision 1.8 (2013/7/17). For both
> boards a
> > >>>>>>>>>>>>>>>>>> BETA revision for the Spectre/Meltdown mitigation is
> > >>>>>>>>>>>>>>>>>> available, but I didn't test that. But please read.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> The third box I realised this problem is a brand new
> Fujitsu
> > >>>>>>>>>>>>>>>>>> Esprimo Q956, also AMI firmware, at V5.0.0.11 R
> 1.26.0 for
> > >>>>>>>>>>>>>>>>>> 3413-A1x, date 05/25/2018 (or 20180525).
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Installing on any kind of HDD or SSD manually or via
> > >>>>>>>>>>>>>>>>>> bsdinstall the OS using UEFI-only boot method on a GPT
> > >>>>>>>>>>>>>>>>>> partitioned device fails. The ASRock boards jump
> immediately
> > >>>>>>>>>>>>>>>>>> into the firmware, the Fujitsu offers some kind of
> > >>>>>>>>>>>>>>>>>> CPU/Memory/HDD test facility.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> If on both type of vendor/boards CSM is disabled and
> UEFI
> > >>>>>>>>>>>>>>>>>> boot only is implied, the MBR partitioned FreeBSD
> > >>>>>>>>>>>>>>>>>> installation USB flash device does boot in UEFI! I
> guess I
> > >>>>>>>>>>>>>>>>>> can assume this when the well known clumsy 80x25 char
> > >>>>>>>>>>>>>>>>>> console suddenly gets bright and shiny with a much
> higher
> > >>>>>>>>>>>>>>>>>> resoltion as long the GPU supports EFI GOP. Looking
> with
> > >>>>>>>>>>>>>>>>>> gpart at the USB flash drives reveals that the EFI
> partition
> > >>>>>>>>>>>>>>>>>> starts at block 1 of the device and the device has a
> MBR
> > >>>>>>>>>>>>>>>>>> layout. I haven't found a way to force the GPT
> scheme, when
> > >>>>>>>>>>>>>>>>>> initialised via gpart, to let the partitions start at
> block
> > >>>>>>>>>>>>>>>>>> 1. This might be a naiv thinking, so please be
> patient with
> > >>>>>>>>>>>>>>>>>> me.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> I do not know whether this is a well-known issue. On
> the
> > >>>>>>>>>>>>>>>>>> ASRock boards, I tried years ago some LinuxRed Hat
> and Suse
> > >>>>>>>>>>>>>>>>>> with UEFI and that worked - FreeBSD not. I gave up on
> that
> > >>>>>>>>>>>>>>>>>> that time. Now, having the very same issues with a new
> > >>>>>>>>>>>>>>>>>> Fujitsu system, leaves me with the impression that
> FreeBSD's
> > >>>>>>>>>>>>>>>>>> UEFI implementation might have problems I'm not aware
> of.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Can someone shed some light onto this?
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> The first thing to check is if the secure boot is
> disabled. We
> > >>>>>>>>>>>>>>>>> do not support secure boot at all at this time.
>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Secure boot is in every scenario disabled!
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> If you have efi or bios version running - you can
> check from
> > >>>>>>>>>>>>>>>>> either console variable value (it can have efi or
> vidconsole
> > >>>>>>>>>>>>>>>>> or comconsole) or better yet, see if efi-version is
> set (show
> > >>>>>>>>>>>>>>>>> efi-version) - if efi-version is not set, it is BIOS
> loader
> > >>>>>>>>>>>>>>>>> running. Another indirect way is to see lsdev -v, with
> device
> > >>>>>>>>>>>>>>>>> paths present, it is uefi:)
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> What are you talking about?
> > >>>>>>>>>>>>>>>> What is the point of entry - running system, loader?
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> sysct machdep.bootmethod: BIOS
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> This makes me quite sure that the system has booted via
> BIOS -
> > >>>>>>>>>>>>>>>> as I'm sure since I've checked that many times. UEFI
> doesn't
> > >>>>>>>>>>>>>>>> work on those systems with FreeBSD. I'm not sure
> antmore, but
> > >>>>>>>>>>>>>>>> I tried also Windows 7 on those mainboards booting via
> UEFI -
> > >>>>>>>>>>>>>>>> and I might recall that they failed also. I also recall
> that
> > >>>>>>>>>>>>>>>> there were issues with earlier UEFI versions regarding
> booting
> > >>>>>>>>>>>>>>>> only Windows 8/8.1 - and nothing else, but the fact
> that Linux
> > >>>>>>>>>>>>>>>> worked confuses me a bit.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> If this ASRock crap (never ever again this brand!)
> doesn't work
> > >>>>>>>>>>>>>>>> at all - who cares, I intend to purchase new server
> grade
> > >>>>>>>>>>>>>>>> hardware. But the more puzzling issue is with the
> Fujitsu,
> > >>>>>>>>>>>>>>>> which I consider serious and from the behaviour the
> Fujitsu
> > >>>>>>>>>>>>>>>> failure looks exactly like the ASRock - Windows 7 works,
> > >>>>>>>>>>>>>>>> RedHat 7.5 works (I assume I can trust the Firmware
> settings
> > >>>>>>>>>>>>>>>> when I disable CSM support, that the Firmware will only
> > >>>>>>>>>>>>>>>> EFI/UEFI capable loader? Or is there a ghosty override
> > >>>>>>>>>>>>>>>> somwhere to be expected?). Also on ASRock disabling CSM
> should
> > >>>>>>>>>>>>>>>> ensure not booting a dual-bootstrap-capable system.
> This said,
> > >>>>>>>>>>>>>>>> on the recent Fujitsu, it seems to boil down to a
> FreeBSD
> > >>>>>>>>>>>>>>>> UEFI-firmware interaction problem, while the ASRock is
> still
> > >>>>>>>>>>>>>>>> under suspicion to be broken by design.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> GPT partitions can never start from disk absolute
> sector 1;
> > >>>>>>>>>>>>>>>>> this is because at sector 0 there is MBR (for
> compatibility),
> > >>>>>>>>>>>>>>>>> sector 1 is GPT table and then sectors 2-33 have GPT
> > >>>>>>>>>>>>>>>>> partition table entries, so the first possible data
> sector is
> > >>>>>>>>>>>>>>>>> 34 (absolute 34). Thats assuming 512B sectors. For
> details
> > >>>>>>>>>>>>>>>>> see UEFI 2.7 Chapter 5.3.1 page 131.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Thanks for the explanation. That implies the installer
> did
> > >>>>>>>>>>>>>>>> right, gpart did also right and therefore there must be
> an
> > >>>>>>>>>>>>>>>> issue with the stuff located within the EFI
> > >>>>>>>>>>>>>>>> partition?
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Ok, so, it is not about UEFI bootcode but BIOS, and if
> we reach
> > >>>>>>>>>>>>>>> BIOS loader at all or not - that is, if the BIOS
> bootstrap is
> > >>>>>>>>>>>>>>> actually caring to read the MBR code and start it, since
> once
> > >>>>>>>>>>>>>>> the MBR code is started, it is all about our code.
>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> I'm getting confused a bit here. Do you mean by "BIOS"
> the CSM?
> > >>>>>>>>>>>>>> or do you mean that specific portion of the UEFI
> firmware, which
> > >>>>>>>>>>>>>> looks for the proper UEFI partition?
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> BIOS as either native or CSM. Note that from boot code
> point of
> > >>>>>>>>>>>>> view the CSM boot *is* BIOS boot, we have no access to UEFI
> > >>>>>>>>>>>>> features.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> The boxes in question, most notably the more recent
> Fujitsu
> > >>>>>>>>>>>>>> Esprimo Q956, refuse booting UEFI, even if properly setup
> (in
> > >>>>>>>>>>>>>> terms of what FreeBSD provides on recent CURRENT) is
> applied and
> > >>>>>>>>>>>>>> CSM is switched off in the firmware. Again: GPT partition
> scheme.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> The system boots properly if a second partition of type
> > >>>>>>>>>>>>>> "freebsd-boot" is applied and bootcode is properly
> applied via
> > >>>>>>>>>>>>>> "gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 2 ada0"
> (ada0
> > >>>>>>>>>>>>>> is the device).
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> btw, you can try to validate the installed boot blocks
> by using
> > >>>>>>>>>>>>>>> recent enough loader (usb or iso) and then you can use
> from OK
> > >>>>>>>>>>>>>>> prompt:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> lsdev provides me with the follwoing informations (CSM
> enabled):
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> OK lsdev
> > >>>>>>>>>>>>>> disk devices:
> > >>>>>>>>>>>>>>        disk0:          BIOS DRIVE C ...
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>                disk0p1:        EFI
> > >>>>>>>>>>>>>>                disk0p2:        FreeBSD BOOT
> > >>>>>>>>>>>>>>                disk0p3:        FreeBSD SWAP
> > >>>>>>>>>>>>>>                disk0p4:        FreeBSD ZFS
> > >>>>>>>>>>>>>> zfs devices:
> > >>>>>>>>>>>>>>        zfs:zroot
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> OK chain disk0
> > >>>>>>>>>>>>>> open failed     (so for disk0p{1-4}.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> OK chain zroot
> > >>>>>>>>>>>>>> failed to read disk (just for completeness)
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> chain command does use only device name (such as disk0: or
> > >>>>>>>>>>>>> disk0p2: ), but not zfs pool as device.  I just found I
> haven?t
> > >>>>>>>>>>>>> ported the code to read the file.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> ??
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> the point for chain command test is to see if the normal
> read and
> > >>>>>>>>>>>>> execute would work, so in your case please try:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> chain disk0:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> As stated above, I did so, and the result is also mentioned
> above,
> > >>>>>>>>>>>> I always get "open failed".
> > >>>>>>>>>>>> This is the same for
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> chain disk0
> > >>>>>>>>>>>> chain disk0p1
> > >>>>>>>>>>>> chain disk0p2
> > >>>>>>>>>>>> chain disk0p3
> > >>>>>>>>>>>> chain disk0p4
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> as already said. CSM is enabled in this case.
> > >>>>>>>>>>>
> > >>>>>>>>>>> sigh? chain command does take device as argument, device
> must always
> > >>>>>>>>>>> end with colon?. in this case, the devil is in details:) as
> I wrote
> > >>>>>>>>>>> above, the command should be:
> > >>>>>>>>>>>
> > >>>>>>>>>>> chain disk0:
> > >>>>>>>>>>>
> > >>>>>>>>>>> The disk0p1: etc will only work when partition boot code was
> > >>>>>>>>>>> installed (which you most likely do not have - the only
> possible
> > >>>>>>>>>>> candidate could be FreeBSD ZFS partition).
> > >>>>>>>>>>
> > >>>>>>>>>> The command "chain disk0:" works as expected (CSM enabled, GPT
> > >>>>>>>>>> partition scheme, but with PMBR bootblock installed and
> freebsd-boot
> > >>>>>>>>>> partition conatining gptzfsboot installed.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> to read pmbr (512B) and execute it. The expected outcome
> would be
> > >>>>>>>>>>>>> that pmbr boot code would browse the GPT, read stage1 from
> > >>>>>>>>>>>>> disk0p2: and execute it; stage1 would detect FreeBSD ZFS
> from
> > >>>>>>>>>>>>> disk0p4: and load and execute /boot/loader. If that will
> happen,
> > >>>>>>>>>>>>> it means the boot code in our stages is just fine, but the
> bios
> > >>>>>>>>>>>>> (CSM) does not load pmbr?.  if thats true, it would mean
> that you
> > >>>>>>>>>>>>> either need to use UEFI boot or need to have some hack to
> fool
> > >>>>>>>>>>>>> the BIOS or just not use GPT on that machine with CSM.
>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> To make it clear here: The only way to boot this box is
> using CSM
> > >>>>>>>>>>>> (as it is the same with the ASRock boards mentioned
> earlier). But
> > >>>>>>>>>>>> my intention is to disable CSM and use a GPT/UEFI
> environment
> > >>>>>>>>>>>> only! And GPT/UEFI doesn't work with FreeBSD, neither with
> > >>>>>>>>>>>> 12-CURRENT, nor 11.2-RELENG.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> It would be nice if this could be fixed. I'm more
> interested in the
> > >>>>>>>>>>>> fix on the recent Fujitsu device than the outdated ASRock
> crap, but
> > >>>>>>>>>>>> if the fix for the Fujitsu Firmware could fix older issues
> as a
> > >>>>>>>>>>>> byproduct, I'd appreciate that.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Kind regards,
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> ok, somehow I have lost that part of the discussion. Well,
> you wrote
> > >>>>>>>>>>> that the UEFI boot fails when the first partition starts
> from sector
> > >>>>>>>>>>> 40 - does it mean you have boot when the partition will
> start from
> > >>>>>>>>>>> some other sector? I think, there is something else going
> > >>>>>>>>>>> on.
> > >>>>>>>>>>
> > >>>>>>>>>> Well, I simply try to describe what I "see" to make things
> > >>>>>>>>>> disambiguous. I'm not familiar with the deeper insights of
> disk
> > >>>>>>>>>> layouts on a binary level. So, you explained to me the
> reason, why
> > >>>>>>>>>> ESP (EGI partition) starts at block 40. I compared that to the
> > >>>>>>>>>> FreeBSD USB flash image FreeBSD provides, but this is another
> story
> > >>>>>>>>>> since the image uses MBR scheme as I figured out.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> What you can do is to see if that firmware will offer you
> EFI shell
> > >>>>>>>>>>> option, from there you can try to start the bootx64.efi
> manually and
> > >>>>>>>>>>> see what error you will get. However, the number 1 cause for
> failing
> > >>>>>>>>>>> to start the bootloader in UEFI is secure boot - we do not
> support
> > >>>>>>>>>>> it and secure boot must be switched off.
> > >>>>>>>>>>>
> > >>>>>>>>>>> However, they seem to claim "The Secure Boot option is
> available in
> > >>>>>>>>>>> the UEFI/BIOS of most if not all ASRock boards. It is
> disabled by
> > >>>>>>>>>>> default.?
> > >>>>>>>>>>>
> > >>>>>>>>>>> Still suggest to double check if thats really the case.
> Also, if the
> > >>>>>>>>>>> bootx64.efi start will fail and no messages are appearing on
> screen,
> > >>>>>>>>>>> then either there is something in firmware logs or you could
> get
> > >>>>>>>>>>> them from trying to start bootx64.efi from UEFI shell.
>
> > >>>>>>>>>>
> > >>>>>>>>>> Since I'm with this problem since 2014 and try from time to
> time, be
> > >>>>>>>>>> ausred that I tried every possible permutationof all
> reasonable
> > >>>>>>>>>> options, even those nonsense, to get rid of that problem.
> > >>>>>>>>>>
> > >>>>>>>>>> I never had any problems with any other UEFI capable
> > >>>>>>>>>> server/workstation firmware so far booting FreeBSD off in
> > >>>>>>>>>> UEFI-native (GPT partition scheme, CSM disabled) so far -
> until now,
> > >>>>>>>>>> when I ran into this Fujitsu ESPRIMO Q956 with the most recent
> > >>>>>>>>>> firmware (as of lat week, week 29 of 2018) having the very
> same
> > >>>>>>>>>> problems.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> I figured out something strange on the Fujitsu - and that is
> the same
> > >>>>>>>>>> with the ASRock boards.
> > >>>>>>>>>>
> > >>>>>>>>>> We/I prepare some USB flash drives to boot a NanoBSD for a
> very small
> > >>>>>>>>>> appliance, but nevertheless, the USB flash device is booted on
> > >>>>>>>>>> Fujitsu servers with UEFI-only configurations. I assume at
> this
> > >>>>>>>>>> point that disabling on the most recent Fujitsu firmwares on
> > >>>>>>>>>> reasonable "new" hardware (not older than three years) will
> disable
> > >>>>>>>>>> any(!) legacy BIOS capabilities. The same is assumed for the
> Fujitus
> > >>>>>>>>>> ESPRIMO Q956. I can not speak for the ASRock A77 Pro4/m boards
> > >>>>>>>>>> mentioned above/earlier, they are from 2012/2013 and "quite
> old".
> > >>>>>>>>>>
> > >>>>>>>>>> The NanoBSD image of ours doesn't have a "freebsd-boot"
> partition.
> > >>>>>>>>>> The partition scheme of the flash device is GPT. The layout
> looks
> > >>>>>>>>>> like this:
> > >>>>>>>>>>
> > >>>>>>>>>> gpart show -l da4
> > >>>>>>>>>> =>      40  15425456  da4  GPT  (7.4G)
> > >>>>>>>>>>    40      2000    1  efiboot0  (1.0M)
> > >>>>>>>>>>  2040   1453584    3  disk1a  (710M)
> > >>>>>>>>>> 1455624      4096    5  disk3  (2.0M)
> > >>>>>>>>>> 1459720  13965776       - free -  (6.7G)
> > >>>>>>>>>>
> > >>>>>>>>>> I created the flash with md, gpart and dd straightforward,
> efiboot0
> > >>>>>>>>>> is the ESP partition and its format/content is created via dd
> > >>>>>>>>>> if=/boot/boot1.efifat of=/dev/da4p1 - I presume this is very
> simple.
> > >>>>>>>>>>
> > >>>>>>>>>> This USB flash device boots(!) successfully (UEFI!) on both
> the
> > >>>>>>>>>> ASRock boards and the Esprimo Q956!
> > >>>>>>>>>>
> > >>>>>>>>>> But any SSD prepared the same way doesn't. Why?
> > >>>>>>>>>>
> > >>>>>>>>>> On the ASRock, I recall having fiddled around with HDD also
> for a
> > >>>>>>>>>> while conatining Windows 7/SP1 and FreeBSD. Windows7 booted,
> FreeBSD
> > >>>>>>>>>> - I can't remember.
> > >>>>>>>>>>
> > >>>>>>>>>> In the lack of proper hardware I'm unable to check whether
> > >>>>>>>>>> USB-attached HDD or SSD will boot or HDD will boot (just in
> case the
> > >>>>>>>>>> local SATA has problems booting UEFI and USB not).
> > >>>>>>>>>>
> > >>>>>>>>>> Kind regards,
> > >>>>>>>>>>
> > >>>>>>>>>> Oliver
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> Am. well. I think the suggestion to test out FAT32 is still
> good one
> > >>>>>>>>> to test. This is because it is known that some vendors do not
> support
> > >>>>>>>>> booting FAT12/FAT16 from HDD (the likely reason is that UEFI
> > >>>>>>>>> specification does not tell which FAT must be supported, and
> only hint
> > >>>>>>>>> about FAT12/FAT16 in context of removable devices).
> > >>>>>>>>
> > >>>>>>>> I prepared yesterday a GTP/ZFS-only 11.2-RELENG on the ESPRIMO
> Q956. It
> > >>>>>>>> took me a time to circumvent the installer and I had to install
> the
> > >>>>>>>> system manually. In that strain, I also "tried" to establish
> the ESP
> > >>>>>>>> with FAT32, as Allen Jude suggested earlier. I didn't find any
> ad hoc
> > >>>>>>>> help how to find out the format (FAT12/16/32) of the ESP, so I
> assume
> > >>>>>>>> when using 12-CURRENT's or 11.2-RELENG's installer with
> AUTO-ZFS and
> > >>>>>>>> GPT (UEFI) (only!) the resulting ESP is FAT12 or FAT16 (300mb by
> > >>>>>>>> default). I also assume, that when dd'ing the /boo/boot1.efifat
> image
> > >>>>>>>> to a partition, the format is FAT, but not FAT32. Therefore, I
> > >>>>>>>> refomatted the manually created ESP (using "gpart add -t efi
> ...")
> > >>>>>>>> using "newfs_msdos -F 32 -b xxx ...". I had to fiddle around a
> bit
> > >>>>>>>> with option -b to fit in a proper format to meet a 512mb ESP -
> I'm not
> > >>>>>>>> sure whether this is the proper option to deal with. When using
> the
> > >>>>>>>> default and only -F32, the size of the partition has to be 4G
> at least
> > >>>>>>>> I assume. Having done that, I copied the the content of
> boot1.efifat
> > >>>>>>>> (mdconfig -t vnode ..., I guess we know the drill ...) to the
> newly
> > >>>>>>>> formatted ESP to /boot/efi/ ...
> > >>>>>>>>
> > >>>>>>>> Having so far no knowledge of how to asure that the created ESP
> is
> > >>>>>>>> FAT32, I assume it is FAT32.
> > >>>>>>>>
> > >>>>>>>> The result is negative on the ESPRIMO Q956. When disabling the
> CSM, the
> > >>>>>>>> box is not willing to boot from SSD with the ESP prepared as
> decribed.
> > >>>>>>>> So, a chance that this might still be due to a misconfiguration
> lies
> > >>>>>>>> now within the -b option of newfs_msdos - if the -b option is
> assumed
> > >>>>>>>> the proper option?
> > >>>>>>>>
> > >>>>>>>> At the moment, the ESP of the Esprimo is subject to changes, if
> you
> > >>>>>>>> wish, but not in size, since it is limited to 512mb.
> > >>>>>>>>
> > >>>>>>>> Thanks and kind regards,
> > >>>>>>>>
> > >>>>>>>> Oliver
> > >>>>>>>
> > >>>>>>> Yea, i was hoping fstyp command would report the FAT type, but
> it does
> > >>>>>>> not (request for feature?:)
> > >>>>>>
> > >>>>>> FYI, the file(1) command is very good at disecting a disk image
> to tell
> > >>>>>> you what the MBR looks like, and at looking at partitions if
> pointed at
> > >>>>>> them with the -s option.  It should be able to detect FAT12/16/32.
> > >>>>>>
> > >>>>>> root_at_x230a:/home/ISO/x # file -s /dev/md2s1
> > >>>>>> /dev/md2s1: DOS/MBR boot sector, code offset 0x3c+2, OEM-ID
> "BSD4.4  ",
> > >>>>>> root entries 512, sectors 1600 (volumes <=32 MB) , sectors/FAT 5,
> > >>>>>> sectors/track 63, heads 1, serial number 0xbd4111ee, label:
> "EFISYS
> > >>>>>> ", FAT (12 bit), followed by FAT
> > >>>>>>
> > >>>>>>>
> > >>>>>>> However, the more annoying idea would be to install some OS
> which will
> > >>>>>>> boot with UEFI on this machine, then copy boot1.efi from freebsd
> to it
> > >>>>>>> (the default program the UEFI will load is
> ESP:EFI/boot/bootx64.efi  in
> > >>>>>>> case of UEFI64 and ESP:EFI/boot/bootia32.efi for EFI32. However,
> we do
> > >>>>>>> not support EFI32.
> > >>>>>>>
> > >>>>>>> note that boot1.efi alone will not do much but printing on
> screen how it
> > >>>>>>> will search for freebsd, but for the purpose of the test it would
> > >>>>>>> suffice
> > >>>>>>> - that would give us confirmed working ESP file system (since
> the other
> > >>>>>>> os would be able to boot) and then we can confirm if boot1.efi
> itself is
> > >>>>>>> OK.
> > >>>>>>
> > >>>>>
> > >>>>> Some new results.
> > >>>>> I installed RedHat 7.5 and inestigated the ESP.
> > >>>>>
> > >>>>> - The ESP starts at block 2048, while FreeBSD's ESP starts at
> block 40.
> > >>>>> - size is both 200mb if installed automatically. I forgit to
> investigate
> > >>>>> the FAT format, but this might be unnecessary as shown further in
> this
> > >>>>> post.
> > >>>>> - RedHat's ESP contains ~ 10 MB of data in two
> > >>>>> folders, /efi/boot, /efi/redhat. copying FreeBSD's BOOTX64.efi over
> > >>>>> RedHat's doesn't change anything, also renaming
> /efi/boot/fbx64.efi of
> > >>>>> RedHat's installation. But renaming /efi/redhat renders RedHat to
> fail the
> > >>>>> boot process on the Fujitsu with the signs of the built-in
> testprogram as
> > >>>>> reported.
> > >>>>>
> > >>>>> I took the liberty and installed 11.2-RELENG again, ZFS only, UEFI
> boot
> > >>>>> only (CSM in firmware disabled, but there is still a
> gptzfsboot-prepared
> > >>>>> partition for later use, just for the record). Booting UEFI-only
> fails as
> > >>>>> reported. On this installation I copied the RedHat ESP completely
> into
> > >>>>> FreeBSD's ESP, renamed /efi/boot/BOOTX64.efi to
> /efi/boot/BOOTX64.efi.rh
> > >>>>> and copied FreeBSD's BOOTX64.efi to /efi/boot.
> > >>>>> The Esprimo Q956 tries then to boot(!) RedHat's kernel. It seems,
> that
> > >>>>> the /efi/redhat folder of the ESP is important, if renamed, the
> whole
> > >>>>> process dies as I reported earlier.
> > >>>>>
> > >>>>> Still unanswered is the question: why is a NanoBSD prepared
> UEFI-only USB
> > >>>>> flash booting with CSM disabled (so asumingly UEFI only then) on
> both
> > >>>>> ASRock and Fujitsu (as described in more detail initially and
> earlier),
> > >>>>> while SSDs fail? Is there a difference? Since FreeBSD boots in
> UEFI mode
> > >>>>> from USB flash prepared as described (straight forward, 800k ESP
> starting
> > >>>>> at block 40, the boot1.efifat image dd'ed onto the partition, UFS
> > >>>>> partition following, no freebsd-boot partition or MBR/PMBR bootcode
> > >>>>> applied ever!), I think BOOTX64.EFI is technically all right.
> There must
> > >>>>> be then an issue with the SATA/SSD/HDD boot pathway.
> > >>>>>
> > >>>>> Hope I could provide some more details, sorry if it sounds
> confusing or
> > >>>>> way too long, but I try to descibe the situation as thorough as
> possible.
> > >>>>>
> > >>>>
> > >>>> OK, this is already good hint. The thing with ESP is that there is
> > >>>> “default” file system tree: EFI/BOOT/BOOT<sysname>.EFI, this is
> noted as
> > >>>> default for *removable* media, fortunately it is usable for hard
> disks as
> > >>>> well, or at least in most cases.
> > >>>>
> > >>>> Now, for non-removable media, the UEFI does provide boot manager
> interface
> > >>>> to define boot entries, and the fact that renaming efi/redhad
> directory did
> > >>>> break the redhat boot, is very loud hint. And this means, this
> system is
> > >>>> probably ignoring efi/boot tree on non-removable media and is
> expecting the
> > >>>> boot manager entry to be created instead.
> > >>>
> > >>> This inplication I'd confirm for the recent Fujitsu ESPRIMO Q956
> firmware
> > >>> (not tested on ASRock Z77-Pro4 firmware).
> > >>>
> > >>>>
> > >>>> UEFI boot manager can be configured /usually/ manually via firmware
> menu,
> > >>>> or by application, such as efibootmgr. The normal approach is to
> create
> > >>>> efi/<vendorname> directory and to copy the application there, then
> create
> > >>>> the boot manager configuration.
> > >>>>
> > >>>> See UEFI specification v2.7, chapter 3 Boot Manager, page 79.
> > >>>>
> > >>>> What is different in FreeBSD case is that the whole interface to
> program
> > >>>> the UEFI Boot Manager is currently being developed, so either it
> has to be
> > >>>> done manually or from some other OS (see
> > >>>> https://wiki.gentoo.org/wiki/Efibootmgr
> > >>>> <https://wiki.gentoo.org/wiki/Efibootmgr> for example, first hit
> from
> > >>>> google:D).
> > >>>
> > >>> Well, thanks for this important hint! FreeBSD 12-CURRENT's and
> FreeBSD
> > >>> 11.2-RELENG's USB flash devices are capable of booting off on
> Fujitsu's
> > >>> ESPRIMO and ASRock's boards. As a note: after "kldload efirt.ko" I
> was able
> > >>> to use the already in FreeBSD present toolset efibootmgr(8) and
> sibblings
> > >>> (the tools do not do anything useful when booted non-UEFI).
> > >>>
> > >>> Mounting the ESP of the harddrive (in my case, ada0p1) to /mnt and
> following
> > >>> the steps in the examples and having created
> /efi/freebsd/BOOTx64.efi as
> > >>> recommended by copy from /efi/boot, let me create a proper boot
> variable.
> > >>>
> > >>> To make things sure, I also applied "efibootmgr -a VARIABLENAME".
> > >>>
> > >>> And ... it worked! Yes, it worked! The ESP is FAT32 formatted, I do
> not know
> > >>> whether this will also work with FAT12/16, I should test this case,
> too.
> > >>>
> > >>> There is a bug in the manpage of efibootmg(8). It does not explain
> the
> > >>> options -d and -p, although they could be "implied" by reading
> carefully.
> > >>> There is now a PR at
> > >>>
> > >>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230080
> > >>>
> > >>> for this issue.
> > >>>
> > >>> So, it's apity that the handbook has no note I could easily find on
> this;
> > >>>
> > >>> Thank you very much for your patience and help!
> > >>>
> > >>> Kind regards,
> > >>> Oliver
> > >>
> > >> yep, efibootmgr does call UEFI RuntimeServices to set up the
> variables, and
> > >> this is only possible when booted UEFI. But glad we finally found the
> root
> > >> cause. It would be good to have HW notes for such cases, it is
> important to
> > >> know that those systems wont boot UEFI from HDD unless the boot
> manager setup
> > >> is done.
> > >>
> > >> rgds,
> > >> toomas
> > >
> > >
> > > This pops up the question how to deal with such HW. We have as a
> institution a
> > > lot of Fujitsu hardware her - from larger servers down to Fujitsu
> ESPRIMO Q956.
> > > The latter one is the first (and so far the only) piece of hardware I
> found
> > > incapable of booting off UEFI within the past 5 years.
> > >
> > > If the "standard" for removeable devices is to boot from
> /efi/boot/bootx64.efi,
> > > than I'd guess it is good luck for FreeBSD that the firmware vendors
> did
> > > fallback to such a mechanism. GRUB/Linux seem to install by default
> their
> > > bootloader into /efi/something/ I'll check on Debian, Suse and CentOS
> so far
> > > soon, the latter probably will, since its the "free" RedHat).
> > >
> > > Anyway, apart from any criticism, I'm glad to have the tools to make
> things
> > > work without using alien help (outside FreeBSD's world!). That is a
> thank you
> > > towards the developers.
> > >
> > > Kind regards,
> > >
> > > oh
> > >
> >
> > The efibootmgr is only relatively recent addition (in illumos we do not
> have yet even
> > access to UEFI RS), so it is only question of time once installer will
> get updated:)
> >
> > But of course there is still an issue about the scenario when the
> install is done in
> > BIOS mode and later switched to UEFI - then the boot manager
> configuration needs to be
> > updated manually (or by some maintenance service like grub2 is calling
> via grub-install
> > script).
> >
> > rgds,
> > toomas
> >
> > _______________________________________________
> > freebsd-current_at_freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-current
> > To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_
> freebsd.org"
>
> Just to add another success on ASRock Z77-Pro4 (800k ESP, FAT12) and
> ASRock Z77-Pro4M
> (300mb ESP, FAT32).
>
> On this firmware, I did not have to define/copy the bootloader
> within /efi/freebd/BOOTx64.efi. It was sufficient to add an EFI variable
> as described in
> the manpage efibootmgr(8).
>
> The only pitfall on this firmware (very old, last functional update 2013,
> Spectre/Meltodown mitigation only May 2018) was that I wasn't able to
> activate variable
> "0000"! Creating
>
> efibootmgr -c -l /mnt/efi/boot/BOOTx64.efi -L FreeBSD-12
>
> which results in "Boot0000"
>
> and followed by
>
> efibootmgr -a 0000
>
> or
>
> efibootmgr -n 0000
>
> resulted in "No such variable" or similar.
>

Yes. that's a bogus sanity check in the code. I've removed it and will
commit in a moment.


> I had to perform the very same task again to gain variable 0001 and then I
> was able to
> "activate" variable 0000. This might be due to the fact the only variable
> defined at all
> was Boot0005 pointing to the most recent USB flash device with 12-CURRENT
> from 2018-07-26
> I just prepared.
>
> Now, also those boxes boot via UEFI (one, 800k ESP with the /efi/boot
> folder, the other,
> 300mb ESP, with a copy /efi/freebsd as I had to do on the Fujitsu ESPRIMO
> Q956 firmware).


OK.

Warner
Received on Fri Jul 27 2018 - 16:03:58 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:17 UTC