[OE-core] Sad story about Shared State

[Richard Purdie]

On Fri, 2011-05-27 at 00:08 +0200, Martin Jansa wrote:
> Good evening,
> 
> sorry for long and pesimistic e-mail, but I hope that someone will point
> me to something I'm doing completely wrong.
> 
> First I was surprised to see how many recipes were rebuilt after this
> change:
> http://git.openembedded.net/cgit.cgi/openembedded-core/commit/?id=aebb9d6599aac683456adf56dc11f8b9f10f25c3
> 
> because [YOCTO #1074] was resolved by bitbake commit:
> http://git.openembedded.net/cgit.cgi/bitbake/commit/?id=139b8a625818225c358a1b8363518d7ed6913188
> and I had rm_work change
> http://patches.openembedded.org/patch/4807/
> during this build I decided to find out what is changing so many sstate
> checksums for such small changes
> 
> so I took another small dbus change from patchwork
> http://patches.openembedded.org/patch/4831/
> and rebuild many recipes again to play with bitbake-diffsig for a while..

[...]

> .. end of story ..
> 
> .. I agree it's "right (TM)" to rebuild not only changed package but also its dependencies ..
> .. I agree it's not possible to differentiate between important library change
>    which can infulence also packages depending on it even if only transitive and
>    cosmetic change improving ie RDEPENDS_${PN} which doesn't influence even direct dependants
> .. but impact of such small change on small shr-lite-image?
>    almost 2 hours on my quadcore AMD Phenom(tm) II X4 965 at 3.4Ghz, 4GB RAM, tmp/work on raid0 (3 SATA2 discs)
> 
> guess how long is runqueue when someone bumps binutils or gcc PR.. it's like DISTRO_PR bump :/
> but almost after every "git pull --rebase".
> 
> this was for our minimal image
> task-shr-feed we have in old OE takes about 2-3 days on our buildhost (dualcore with 2GB ram)
> for 1 arch, we (SHR) are supporting 3 arm architectures 
> armv4t (om-gta01, om-gta02), armv6-novfp (htcdream), armv7a (palmpre, palmpre2, nokia900)
> so currently it takes 10-14 days to rebuild feed after DISTRO_PR change..
> 
> I'm not sure if we can afford our feeds to track oe-core/meta-oe HEADs 
> like we did with old OE :(.

It should still be possible for OE-Core to behave like "old OE".

> Is it expected use-case to base layers only on released version of 
> oe-core/meta-oe and never touch recipes which are higher in dependency tree?

It is expected that as we get a more stable and fully featured core that
people should be able to base off released versions of it rather than
needing to track master the whole time.

> Is it possible to disable sstate and do DISTRO_PR bumps manually when 
> distro maintainder decides it's really needed and buildhost is free 
> for long enough to populate feeds?

What do you mean by disable sstate as there are several different
elements to it.

Which BB_SIGNATURE_HANDLER are you using? It sounds like you're using
basichash but want the behaviour generated by basic (the current
default). Nobody is forcing you to use basichash.

As you mention, there are benefits and drawbacks either way.

Cheers,

Richard