[OE-core] RFC: Braindump on Bootloaders, Image Types, and Installers

[Koen Kooi]

Op 8 jun. 2012, om 08:16 heeft Saul Wold het volgende geschreven:

> On 06/07/2012 09:11 AM, Koen Kooi wrote:
>> 
>> Op 6 jun. 2012, om 18:47 heeft Mark Hatle het volgende geschreven:
>> 
>>> I don't have specific comments below, but I can share some of what I will be working on for my company in the next 6 months or so.
>>> 
>>> I will be working on moving the image generation: filesystem construction, filesystem ->  image generation, and similar into a framework that can reside externally of OE-core/bitbake.
>>> 
>>> The idea is that we should have a single framework that can construct the filesystem from the package feeds, construct the images (using the parameters and more that you list below), and then eventually deploy these images onto the target system.
>>> 
>>> Having this external of oe-core will allow it to be invoked by OE-core, as it is today, as well as for on-target installers, people who want to start w/ the package feeds, etc.
>> 
>> Chris Larson mentioned this idea during lunch at the last ELC. For angstrom we already put heavy emphasis on the narcissus tool for users to generate their images and lately I have been needing root permissions for various ext4 tweaks which oe-core/bitbake can't do (and really shouldn't).
>> 
> Hmm, more info here, please.
> 
> I am looking at moving OE-Core to use ext4 by default, is there something that will break if we do that?  It seems like it's the right time to do that based on where maintenance and support of ext3 and ext4 are.

tl;dr version: a real ext4 partition requires mkfs.ext4, genext2fs + tune2fs + fsck is flaky and won't give you the actual ext4 benefits.


Some background: my ext4 quest was started to make the system feel faster when using cheap kingston sd cards. I started out with thinking "ext4 is just ext2 with some extra bits", so I did the genext2fs + tune2fs thing:

case "${ROOTFSTYPE}" in
		ext3)
				genext2fs -z -N $FS_NUM_INODES -b $FS_SIZE_BLOCKS -d ${IMAGE_ROOTFS} ${LOOPDEV_FS}
				tune2fs -L ${IMAGE_NAME} -j ${LOOPDEV_FS}
				;;
		ext4)
				genext2fs -z -N $FS_NUM_INODES -b $FS_SIZE_BLOCKS -d ${IMAGE_ROOTFS} ${LOOPDEV_FS}
				tune2fs -L ${IMAGE_NAME} -j -O extents,uninit_bg,dir_index ${LOOPDEV_FS}
				;;
		*)
				echo "Please set ROOTFSTYPE to something supported"
				exit 1
				;;
esac

The above plays nice with bitbake and pseudo and according to all the wikis yields an ext4 system. And as a bonus the u-boot ext2 parser can load a uImage from it. It has however a few downsides:

1) you need to run fsck.ext4 to init the dir_index
  1a) fsck.extX is stupid and will not always treat that ext4 system as an ext4 system. If you have an fsck .conf file in /etc/ you've already lost. I had it working on my build machine, but after an update to e2fsprogs it stopped working, it only ran in ext2 mode. Ironically, the only systems that will reliably treat that partition as ext4 are my OSX systems where I compiled and installed e2fsprogs myself.
2) the big ext4 feature 'extents' will only work on files created *after* that fsck.

So you end up with an ext3 system that needs an fsck to actually work and has none of the ext4 features out of the box.

What I have been doing to get around this is totally screwed up:

The initial creation of the SD card:

* generate the multipartition image with OE
* mkfs.ext4 the second partition, erasing what OE put there with genext2fs + tune2fs
* tar xjf the rootfs.tar.bz2 to the second partition
* dd image to sd card

And for all the updated images the past few weeks:

* opkg update ; opkg upgrade on the target
* rm /mnt/loop/home/root/.[A-z]* /mnt/loop/etc/dropbear/* -rf on the host
* dd /dev/sdX | pv -s 3484M > sd.img

The only advantage of this is that upgrade paths must work and I never run into changes-without-PR-bump problems because the angstrom uploader filters out duplicate filenames before uploading. But for the rest: completely screwed up.

regards,

Koen