[oe] Checksums in Bitbake

[Chris Larson]

On Wed, Mar 24, 2010 at 8:15 AM, Frans Meulenbroeks <
fransmeulenbroeks at gmail.com> wrote:

> Interesting ideas.
> I need to let this digest a little  bit.
>
> Some initial thougths
> The checksum should also depend on the checksum of the underlying
> packages. E.g. if A depends on B and the checksum of B changes it
> should trigger a rebuild of A.
>

I don't think this is a very good idea, personally.  As an option, perhaps,
but we do things the way we do for a reason, just because a dep of mine is
rebuilt doesn't automatically require that I be rebuilt.  I'd suggest moving
to an alternative which encodes the library ABI and incorporates that into
the hashes of things that depend upon it, but we can certainly do what you
want as an optional feature.

A first crude approach would be to have a hash of the concatenation of
> the unfolded recipe (so with all includes/requires expanded) and the
> hashes of the recipes it depends on). Of course this is very rough as
> even changing whitespace in a recipe will lead to a recompile.
> A different approach would be to let it depend on PV + PR. That'll put
> the developer in control (with all related issues, like the developer
> not bumping PR).
> And yet a different one would be to use variables and functions from the
> recipe.
>
> I have mixed feelings on whether checksums also would depend on global
> vars (e.g. code generated by the classes or variables in e.g.
> local.conf).
> On the one hand it seems pretty neat, on the other hand I worry about
> performance (calculating the checksum).
>

Global variables should absolutely be included, imo.  The reason for going
with a blacklist rather than a whitelist approach is to, as richard says,
make it less error prone.  It ensures that the failure mode is something
being rebuilt, rather than using possibly incorrect binaries.  I'd rather it
take a bit longer to build than result in questionable output.  If
calculating the checksum time becomes a concern, which I doubt, you could
hash the configuration metadata at ConfigParsed time and incorporate that
hash into the hash generated of the recipe.  This could increase the
likelihood of collisions, but I'm not too worried.  Let's get things
working, and determine the bottlenecks at that point.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics