[bitbake-devel] (EXT) Re: Forcing the rebuild of dependent tasks across recipes

Matthias Schiffer matthias.schiffer at ew.tq-group.com
Mon Nov 25 09:58:06 UTC 2019


On Fri, 2019-11-22 at 11:15 -0600, Mark Hatle wrote:
> 
> On 11/22/19 9:09 AM, Matthias Schiffer wrote:
> > Hi all,
> > 
> > is there a robust way to enforce the automatic rebuild of dependent
> > tasks across recipes? I'm having the following issue:
> > 
> > - Recipe my-core-image has an output file that is copied to the
> > deploy
> > dir. The output file is not fully reproducible, so cleaning the
> > recipe
> > and rebuilding it will yield a different file even through the
> > sstate
> > signature of the recipe is unchanged (the file I'm interested is
> > DMVerity hash data that I generate as an additional filesystem
> > type)
> > 
> > - Task do_compile of recipe my-initramfs-scripts depends on my-
> > core-
> > image:do_image_complete. I need to enforce rebuilding of this task
> > whenever the deployed files of my-core-image have changed (the
> > recipe
> > uses the "root hash" generated by DMVerity as input, so that the
> > filesystem can be verified on mount) - and of course also of all
> > dependent tasks like the initramfs image and FIT kernel image
> > 
> > I've experimented with things like file-checksums and
> > BB_DONT_CACHE,
> > but everything I've tried fails when the rebuilds of my-core-image
> > and
> > my-initramfs-scripts are supposed to happen in the same run of
> > Bitbake:
> > 
> > - file-checksums only seems to be taken into account at initial
> > recipe
> > parse time, not when the task I'm interested in is actually
> > executed? I
> > need to run bitbake twice to get the correct results.
> > - When I use an anonymous Python function to put the root hash I'm
> > interested in into a variable, Bitbake complains about
> > nondeterministic
> > metadata.
> 
> There is a way of doing this, I've had to do it in the past.
> 
> The anonymous python function needs to product the same output on
> each and every
> run.  If it doesn't this will cause the non-deterministic
> behavior.  (It can't
> rely on another items to finish building for instance.)
> 
> So you end up having to have a recipe (with a file-checksum and/or
> anon python)
> that sets up variables that will keep the hash up-to-date for THAT
> recipe.. then
> your image (or other) recipes need to depend on that one.

Hello Mark,

I'm afraid I don't follow. How does doing something in another recipe
fix the non-deterministic behaviour?

I'd think that any solution based on the task signatures will have the
same issue - that a hash correctly reflecting the inputs that go into
my initramfs task would depend on the build results of my core image
task, which is not available at recipe parse time.

I think I'll have to go with nostamp after all - the unnessesary
rebuilds it causes in this case are ugly, but rather negligible in
terms of build time.


Thanks,

Matthias



> 
> Unfortunately I no longer have access to the example where I had to
> do this in
> the past, or I could try to give you something more concrete to look
> at.
> 
> --Mark
> 
> > 
> > Using nostamp would be an option, but obviously an ugly one.
> > 
> > I'm currently on Yocto Thud / Bitbake 1.40.
> > 
> > 
> > Thanks in advance,
> > 
> > Matthias
> > 



More information about the bitbake-devel mailing list