[OE-core] RFC: Braindump on Bootloaders, Image Types, and Installers

[Darren Hart]

All,

There has been a lot of discussion scattered across the lists and
bugzilla regarding bootable image types. There are a variety of
different situations, each with specific requirements. Our current set
of image types are showing their age. Here I collect what I've seen in
an attempt to start a consolidated discussion to help arrive at a clear
vision of where we want to be, and to define tasks that we can
distribute across various people. If you are on CC, I intend to include
you in that list of people ;-)

Current issues:
* hddimg types fail to boot on many intel platforms [3]
* USBZIP image creation is slow and not integrated into the bitbake
  process
* dosfstools is stuck at an old version to avoid requiring GPLv2 to
  build images
* no support for kernel+initramfs as a single blob [9]
* can't create partitioned images as the tools require root access
* hybrid ISO images are more universally bootable, but don't provide
  persistent storage even on USB sticks.
* different boot loaders are used on the live images and the installed
  images
* boot parameters are not preserved across installation [0]
* the installer is very limited to where it can install
* the installer doesn't work for EFI images [1]
* the hddimg will load the first rootfs it finds, even if it isn't the
  one on the hddimg boot media
* the boot prompts are not self descriptive
* EFI, PCBIOS, ISO all have subtly different boot requirements which
  complicate the bootimg recipe and associated classes.
* custom multi-partition images are not supported [4]
* Can't build the image and the bootloader as different arches [5]
* Various issues with unionfs [7] and chroot [8]

... and that's just for x86...

Where I envision images going:

* Create the follwing image types:
  * Live Installer
    - this is the merger of hddimg, ISO, and ISOHYBRID
    - non-persistent storage
    - intended for demonstration and installation
    - can be written directly to USB sticks or CDs
  * Disk Image
    - create partitioned images (not just volume images)
    - intended to be written to USB sticks, MMC drives, or SATA disks
    - may require a script to assemble the image for specific target
      devices.
  * Initramfs
    - Linux kernel and initramfs as a single image
    - this will require some significant workflow changes as images are
      generated after the kernel and the kernel needs the rootfs to
      build the combined image.

As for boot loaders. In x86 land, we have syslinux, efilinux, grub, and
grub-efi. We use syslinux for the hddimg and grub after installation,
which accounts for some of the boot parameters being lost in the
transition. I feel we would benefit from unifying the bootloader across
image types and presenting a consistent interface. For the core, I plan
to adopt syslinux everywhere [6]. Syslinux supports ISO images, EXT
filesystems, FAT filesystems, and will support EFI soon enough. It also
has the option for a menu system and basic graphics. With this, we can
greatly improve the initial experience and keep that consistent across
the various image types, as well as after installation.

EFI merits a special note here. While working with an EFI enabled
system, it often makes sense to work from the EFI shell for a variety of
reasons. As such, packaging and including efibootmgr to manage boot
targets in the EFI firmware is something we should be providing in
addition to the common use case of a boot/install boot loader. Our
current EFI images frequently do not boot automatically, despite using
the "boot/EFI/boot[ARCH].efi" path for the payload as specified in the
EFI specification. This may be do to the media not being ready in time
(USB can be awfully slow). Some additional research is clearly necessary
to properly support EFI in our images, and likely some hacks to work
around buggy firmware [2]. We should be making friends with Matthew Gerrett.


References
[0] https://bugzilla.yoctoproject.org/show_bug.cgi?id=2426
[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=1919
[2] https://bugzilla.yoctoproject.org/show_bug.cgi?id=1950
[3] https://bugzilla.yoctoproject.org/show_bug.cgi?id=1763
[4] https://bugzilla.yoctoproject.org/show_bug.cgi?id=1780
[5] https://bugzilla.yoctoproject.org/show_bug.cgi?id=2073
[6] https://bugzilla.yoctoproject.org/show_bug.cgi?id=1635
[7] https://bugzilla.yoctoproject.org/show_bug.cgi?id=2343
[8] https://bugzilla.yoctoproject.org/show_bug.cgi?id=2064
[9] https://lists.yoctoproject.org/pipermail/yocto/2012-May/008633.html
-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel