[oe] STAGING_BINDIR

[Richard Purdie]

On Fri, 2006-11-24 at 14:13 +0000, Richard Purdie wrote:
> STAGING_DIR = "${TMPDIR}/staging"
> STAGING_BINDIR = "${STAGING_DIR}/${BUILD_SYS}/bin"
> STAGING_LIBDIR = "${STAGING_DIR}/${HOST_SYS}/lib"
> STAGING_INCDIR = "${STAGING_DIR}/${HOST_SYS}/include"
> STAGING_DATADIR = "${STAGING_DIR}/${HOST_SYS}/share"
> STAGING_LOADER_DIR = "${STAGING_DIR}/${HOST_SYS}/loader"
> STAGING_FIRMWARE_DIR = "${STAGING_DIR}/${HOST_SYS}/firmware"
> [...]
> My proposal is therefore to use:
> 
> STAGING_BINDIR = "${STAGING_DIR}/${BUILD_SYS}/bin/${HOST_SYS}" and have
> native.bbclass override this with ${BUILD_SYS}/bin
> 
> We'd then add both ${BUILD_SYS}/bin and ${BUILD_SYS}/bin/${HOST_SYS} to
> the PATH.
> 
> Does anyone see a better way to handle this?

I've done some experimentation and had a brief discussion with mickeyl
on irc. I'm going to update this proposal with the result of this and
some real world testing.

We need three staging binary directories:

1. STAGING_BINDIR = "${STAGING_DIR}/${HOST_SYS}/bin"

HOST_SYS binaries (say arm) go here. This is *not* in PATH. We never
want to run arm binaries on x86 for example.

2. STAGING_BINDIR_CROSS = "${STAGING_DIR}/${BUILD_SYS}/bin/${HOST_SYS}"

Binaries that run on BUILD_SYS (x86) but produce results for HOST_SYS
(e.g. gcrypt-config or *-config) go here. The path might change into
somewhere in CROSS_DIR at some point once I work out more about the
structure there but that is easy if everything references the right
variable.

This is in PATH before 3. below.

3. STAGING_BINDIR_NATIVE = "${STAGING_DIR}/${BUILD_SYS}/bin"

Anything generated by the -native builds (to run on x86) goes here. This
appears in PATH, after STAGING_BINDIR_CROSS.

In native.bbclass we set STAGING_BINDIR = STAGING_BINDIR_NATIVE =
STAGING_BINDIR_CROSS. This means we have the same path twice in PATH but
apart from that several nice things happen. Native programs access
native *-config files in the NATIVE bindir for example. Also, this means
-native packages need no changes to work with the new variables! :).


How do we go about this change? I propose to start to split the current
references to STAGING_BINDIR into these three types. Initially we update
binbake.conf to say:

STAGING_BINDIR = "${STAGING_DIR}/${HOST_SYS}/bin"
STAGING_BINDIR_NATIVE = "${STAGING_DIR}/${HOST_SYS}/bin"
STAGING_BINDIR_CROSS = "${STAGING_DIR}/${HOST_SYS}/bin"

which means nothing breaks whilst we rename variables. I have a patch
which starts to do this.

As some point we then throw a switch and try out the new values I
mentioned above which will work with multimachine. This should give us a
structured transition.

This has important future implications for packaged staging. It means
installed packages can put arm binaries in STAGING_BINDIR and we won't
corrupt PATH with unexecutable binaries. It does mean we will need to
mark packages with cross binaries to install so packaged staging can
either be somehow supplemented with a cross package to install (or the
staging package is different from the installable package). If we have
references to STAGING_BINDIR_CROSS in the affected packages, this makes
that process easier.

My testing did show some interesting cases we're going to have to catch.
Taking libidl as an example, its do_stage does a "make install bindir=
${STAGING_BINDIR}". This is fine but libIDL-config-2 is a cross script
and should be in STAGING_BINDIR_CROSS and is installed into the wrong
place. My local copy has do_stage updated to read:

cp ${STAGING_BINDIR}/libIDL-config-2 ${STAGING_BINDIR}/libIDL-config-2.orig
cat ${STAGING_BINDIR}/libIDL-config-2.orig | sed -e 's:${includedir}:${STAGING_INCDIR}:' > ${STAGING_BINDIR}/libIDL-config-2

if [ "${STAGING_BINDIR}" != "${STAGING_BINDIR_CROSS}" ]; then
   mv ${STAGING_BINDIR}/libIDL-config-2 ${STAGING_BINDIR_CROSS}/libIDL-config-2
fi

(the file has to handle the native and non-native cases, hence the if)

As always, I'm open to alternative suggestions, help and assistance.
Does anyone have a problem with the changes to at least use the three
different variable names? That shouldn't break anything whilst we set
them all equal in bitbake.conf, yet lets us clean up the metadata.

Cheers,

Richard