[Openembedded-architecture] Recipe Specific Sysroots

[Richard Tollerton]

Richard Purdie <richard.purdie at linuxfoundation.org> writes:

> I've made a proof of concept available on this branch:
> 
> http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=rpurdie/wip-rss

Yay!

> The first is that there is big difference between a sysroot per task
> and a sysroot per recipe. A sysroot per task would be fully
> deterministic, but very heavy. A recipe specific sysroot is not
> entirely deterministic since each task requires different
> dependencies. For example, do_fetch might need subversion-native,
> do_patch, quilt- native and then do_configure, DEPENDS.

OTOH, I would expect do_fetch and do_patch specifically to be extremely
common across all recipes. It almost makes more sense to have a separate
sysroot for do_fetch/do_patch/etc than to use the same sysroot as used
in the build.

> For performance, its always been envisaged that the recipe specific
> sysroots would consist of hardlinks to minimise disk usage and IO and
> to try and minimise the performance impact of this change. Currently
> sstate is relocated into position once. With more sysroots, we need to
> relocate the code into each sysroot. This is particularly try with
> things that contain large numbers of references to their installed
> location. quilt-native is particularly bad, as are native .la, .pc and
> -config files.

Possibly stupid asking-for-a-pony question: are we approaching a level
of complexity here where switching to a chroot-like approach might be
more maintainable than relocations? i.e., create a new root dir for each
task/recipe, bind-mount the sysroot and relevant source/build components
into it, then chroot into that to do the build. Or fake a chroot out via
LD_PRELOAD.

This also faintly reminds me of Google's cloud build tools [1], wherein
a large amount of development complexity winds up getting hidden behind
a FUSE fs. One could avoid creating sysroots on disk in the first place,
instead populating and relocating them on-the-fly from sstate or
something. That could also provide a way to solve dependency rebuild
problems.

[1] http://google-engtools.blogspot.com/2011/10/build-in-cloud-distributing-build.html

> * There are problems with anonymous python execution ordering. If
> depends get added after the staging.bbclass anon python, dependencies
> are missed and they don't get installed into the sysroot. We could run
> the population funciton on every task but that seems wasteful.

Doing that would be on the critical path to per-task sysroots, no?