[OE-core] [PATCH] wic: add --reserved-size wks option

[Maciej Borzęcki]

On Tue, Oct 18, 2016 at 11:27 AM, Ed Bartosh <ed.bartosh at linux.intel.com> wrote:
> On Tue, Oct 18, 2016 at 11:24:14AM +0200, Maciej Borzęcki wrote:
>> On Tue, Oct 18, 2016 at 10:38 AM, Ed Bartosh <ed.bartosh at linux.intel.com> wrote:
>> > On Tue, Oct 18, 2016 at 10:37:22AM +0200, Maciej Borzęcki wrote:
>> >> On Tue, Oct 18, 2016 at 9:31 AM, Ed Bartosh <ed.bartosh at linux.intel.com> wrote:
>> >> > On Mon, Oct 17, 2016 at 04:46:00PM +0200, Maciej Borzęcki wrote:
>> >> >> On Mon, Oct 17, 2016 at 3:22 PM, Ed Bartosh <ed.bartosh at linux.intel.com> wrote:
>> >> >> > Hi Maciej,
>> >> >> >
>> >> >> > There is already --size and --extra-space options.
>> >> >> > Can we get the same or similar result by just using them? Do we really
>> >> >> > need new option for similar purpose?
>> >> >>
>> >> >> --reserved-size serves a different purpose, it establishes an upper
>> >> >> bound on the size of a partition during layout. Unlike
>> >> >> --size/--extra-space does not depend on the size of the filesystem
>> >> >> image.
>> >> >>
>> >> >> For instance, assume I'm creating an image for SD card/eMMC with a
>> >> >> fixed partition layout (something simple: boot partition, primary &
>> >> >> secondary rootfs partitions, some data partition). Because future
>> >> >> system updates are delivered as filesystem image, I want to make sure
>> >> >> that there is exactly xxx MBs for my current and future rootfs images
>> >> >> (regardless of current image size). Neither --size nor --extra-space
>> >> >> can do that. I could use, say `--size 200 --overhead-factor 1`, but
>> >> >> this will needlessly create a 200MB rootfs image and if I happen to
>> >> >> cross the 200MB boundary I will not get an error.
>> >> >>
>> >> >> I had a private patch that added --fixed-size to enforce --size, but
>> >> >> that would still end up creating filesystem image to fill the whole
>> >> >> space.
>> >> > I didn't get the difference between enforcing partition size and below
>> >> > implementation. Can you elaborate a bit?
>> >>
>> >> `--fixed-size` was something that I had added to my fork back in 2014,
>> >> even before `--overhead-factor` came in. The problem is that depending
>> >> on a project you might want to have more control over how partitions
>> >> are laid out, or even need to have a fixed layout. Adding
>> >> `--fixed-size` would had a similar effect to what `--overhead-factor
>> >> 1` does right now. Combined with `--size` would ensure that rootfs is
>> >> say, 200MB large. The downside was that wic would actually create a
>> >> 200MB rootfs, something that is not really necessary. In fact, I only
>> >> wanted to have 200MB gap so that I have some spare space for future
>> >> updates (where update is just a rootfs image you dd to the partition).
>> >>
>> > Thanks for the explanations. Now I got it - reserved size is not a part
>> > of partition, it's a gap between partitions.
>>
>> I might have not been clear enough when explaining. It's not a gap,
>> it's just a container of size --reserved-size listed in MBR/GPT.
>> There's probably a filesystem inside but not necessarily.
>> Graphically it looks as like this:
>>
>>                          --reserved-size
>>                       |----------------------|
>>                       v                      v
>>     +-----------------+----------------------+---------------------+
>>     |..... stuff .....|xxxxxxxxxx            |..... stuff .........|
>>     +-----------------+----------------------+---------------------+
>>                       ^         ^            ^
>>                       |---------|------------|
>>                        --size    --extra-space
>>
>>
> Ah, I'm wrong again. It's a partition size limit, but it's not necessary
> formatted, right? It's only formatted if size == reserved_size.
>
>> >
>> > What's the advantage of creating unusable gap over creating partition of
>> > the same size that can be used?
>>
>> Just convenience.
>>
> What's the convenience of having extra space on partition that can't be
> used for data over having it formatted and used?
>
>> > Even if that space is not needed it doesn't harm to have it, does it?
>>
>> I have not seen any negative side effects.
>>
> I do. If user needs that reserved space it's impossible to get it without
> reformatting partition. The free space exists, but can't be used.

That's not the point and is not aligned with use case I'm trying to solve.

My case is rather simple, I'm creating an image for SD card that is
deployed in the field. In that particular case, there's a primary and
a secondary (aka. active and inactive) rootfs partitions that are
switched whenever a system update comes in. The update is a file
system image that is copied over to the inactive partition, followed
by a system reboot.

What I need is the ability to set a certain size of a partition (say
100MB), regardless of current rootfs size (which may be, say 70MB).
The remaining unused space sets an upper limit on how much the rootfs
may grow in the future (in this example case, it's 30MB). RIght now
the best I can do is to describe a partition like this: `part /
--source rootfs --size 100MB --overhead-factor 1`, hoping that if
rootfs grows beyond 100MB I will somehow still be able to catch that
and that the future images remain size compatible.

The resulting filesystem inside the partition is larger than what
IMAGE_CMD (ex. IMAGE_CMD_ext4) would give me, because of explicit
--size in kickstart. I would prefer to have something comparable in
size just to avoid later surprises, what implies using defaults.
However, using defaults, means that I cannot control the layout
because it will likely change each time rootfs size gets changed.
There is no `--fixed-size` or other option to enforce specific size.

Summing up, a simple use case that cannot be currently solved using wic.

BTW. actually we're missing an ability to enforce --size (i.e.
--fixed-size?) and a method passing an explicit partition offset
inside the disk image (something useful for `--source rawcopy
--no-table` partitions, currently solved with `--align`).

Cheers,
-- 
Maciej Borzecki
RnDity