[bitbake-devel] Forcing the rebuild of dependent tasks across recipes

[Richard Purdie]

On Fri, 2019-11-22 at 16:09 +0100, Matthias Schiffer wrote:
> is there a robust way to enforce the automatic rebuild of dependent
> tasks across recipes? I'm having the following issue:
> 
> - Recipe my-core-image has an output file that is copied to the
> deploy
> dir. The output file is not fully reproducible, so cleaning the
> recipe
> and rebuilding it will yield a different file even through the sstate
> signature of the recipe is unchanged (the file I'm interested is
> DMVerity hash data that I generate as an additional filesystem type)
> 
> - Task do_compile of recipe my-initramfs-scripts depends on my-core-
> image:do_image_complete. I need to enforce rebuilding of this task
> whenever the deployed files of my-core-image have changed (the recipe
> uses the "root hash" generated by DMVerity as input, so that the
> filesystem can be verified on mount) - and of course also of all
> dependent tasks like the initramfs image and FIT kernel image
> 
> I've experimented with things like file-checksums and BB_DONT_CACHE,
> but everything I've tried fails when the rebuilds of my-core-image
> and
> my-initramfs-scripts are supposed to happen in the same run of
> Bitbake:
> 
> - file-checksums only seems to be taken into account at initial
> recipe
> parse time, not when the task I'm interested in is actually executed?
> I
> need to run bitbake twice to get the correct results.
> - When I use an anonymous Python function to put the root hash I'm
> interested in into a variable, Bitbake complains about
> nondeterministic
> metadata.
> 
> Using nostamp would be an option, but obviously an ugly one.
> 
> I'm currently on Yocto Thud / Bitbake 1.40.

If you look behind the scenes at how "bitbake x -f" works, it writes
.taint files into ${STAMPS}.

If A depends on B and B taints A's stamp each time it runs, it will
rerun that task and all tasks that depend upon it. See
bb.build.write_taint(). That function is not designed to be called
externally so you may need your own version. 

Ugly, but your problem is ugly. We don't have a standard mechanism as
we don't really want to encourage this.

Cheers,

Richard