[Openembedded-architecture] Sharing sstate cache and detecting build host paths in -native recipes

[Richard Purdie]

On Wed, 2017-03-29 at 11:06 +0200, Phil Wise wrote:
> Recently we've started using SSTATE_MIRRORS to share intermediate
> build output between machines, and have run into a few issues where
> -native recipes end up baking in hard-coded paths to their build
> hosts. One example would be [1], but many Linux executables hard-code 
> paths to resources, and I would consider it to be 'normal' Linux
> behavior.
>
> So far we've fixed this in a whack-a-mole fashion by either changing
> code to pass in paths at run time, or by making the recipe explicit
> about it's dependency:
> 
> SYSROOT_DIR = "${STAGING_DIR_TARGET}"
> SYSROOT_DIR_class-native = "${STAGING_DIR_NATIVE}"
> do_configure[vardeps] += "SYSROOT_DIR"
> 
> 
> What is the best way to solve this permanently?

We've been doing this for a while on the yocto project autobuilders and
so far in practise, we've not found there to be huge amounts of issues.
We've been approaching it with fixing issues if/as/when we see them.
The recently change to use RSS in master basically makes it a hard
requirement for things to relocate and if they don't errors are much
more obvious so we do have a commitment to making this work.

We do have a few tricks to fix things. If you grep oe-core for
"create_wrapper", you'll see recipes like git, perl, rpm and so on use
this trick to set environment variables in a wrapper script which then
allows them to operate correctly. We have patches sources in some cases
to allow ensure we have the right environment variables available.

The trick mentioned in #9883 is an extension to this, one we already
use in nativesdk for some binaries where we move symbols into their own
section and rewrite this at installation time. This is more invasive
than the wrapper scripts and likely not upstreamable though so we only
use it for key problems like gcc/glibc right now.

We also pass parameters into some of the tools we use to ensure they do
use the right paths.

Hopefully you're only running into a small number of problems and once
fixed, you should be good.

Cheers,

Richard