[OE-core] RFC: Secondary Toolchain

[Mark Hatle]

On 10/4/12 3:36 PM, Khem Raj wrote:
> On Thu, Oct 4, 2012 at 11:02 AM, Mark Hatle <mark.hatle at windriver.com> wrote:
>> We have an issue where we'd like to have an alternative toolchain
>> (assembler, linker, compiler) available for our customers to selectively
>> use.  However, before we just go off and implement something, I'd like at
>> least some basic consensus on what the best practice is for doing this work.
>> Below is my attempt at an early proposal.
>>
>> Background
>> ----------------
>>
>> Many companies have commercial / highly optimized toolchains for specific
>> purpose, such as ICC from Intel, LLVM, ARM specific toolchain, etc..  For
>> (potentially) better performance with some applications a mechanism to both
>> provide the hooks for the alternative toolchain as well as a way to active
>> it per-package is desired.
>>
>> This work assumes that the third party toolchain is generally compatible
>> with the idea of sysroots, linking to the libc provided by OE, and generally
>> compatible with GNU conventions.
>>
>> However, as part of the third party toolchain, it may not be GNU compatible.
>> This means many Open Source applications simply may not work with this
>> toolchain.  That means that we need to have a way for a toolchain to
>> blacklist (or whitelist) specific recipes.  This way only supported
>> components can be built by the user, avoiding numerous complaints.
>>
>> Currently OE has a method to generate an SDK based on the GNU toolchain.  We
>> would like to be able to also export the external toolchain along with the
>> SDK, effectively providing both the GNU toolchain and the third party
>> toolchain using the common sysroot.
>>
>> We need a way to active the third party toolchain on a per-package basis.
>
> Per package bases is a bit iffy from my POV if you consider the compiler runtime
> and C library runtime and so forth. Is it intended to just compile and
> be able to
> cope with the default runtimes on images or do you also intend to figure out
> ways for multiple runtimes to live together.

This is only one runtime.  You have multiple compilers all capable of producing 
software compatible with the same ABI.  The default (oe) compiler is used, 
unless otherwise configured.  The alternative(s) are used for optimization of 
various items.  I mentioned ICC, because it's one that I know today people are 
using and it is capable of replacing gcc in many applications.

The issue is that I want to enable someone to use an alternative compiler, but I 
don't want to do anything beyond enabling it if it doesn't work.

>> This activation will need to use the existing sysroot, but be able to pass
>> different C, C++, LD, AS and other flags as specified by the third party
>> toolchain.
>
> that sort of answers the above. So we are assuming that toolchain that
> can interoperate
> will be using GNU runtime
>
> There are cases where you might want something like ELLCC which avoids
> complete GPL runtime e.g.
> I think it could be divided into runtime  + tools where runtime lives
> in a different layer but
> essentially offering to let people use builtin C library and runtime
> may not be a bad choice

When the runtime changes, I consider that to be a different abi and outside the 
scope of this.  Thats where the machine/multilib/etc configuration items come 
into play.  (I.e. some architectures support an alternative compiler for bare 
metal applications -- in those cases the canonical arch is no longer 
-linux[-gnu]... so they would simply be treated as a different architecture.)

>
>>
>> Finally third party toolchains should be implemented as layers that can
>> easily plug into OE.
>
> agreed. I think we need to document the interfaces in OE-Core for this
> too so that
> people can rely on the interface. Otherwise it will be constant pain
> for keeping those
> layers working together.

Exactly.. I don't want people coming to me saying I want XYZ compiler, but it's 
implemented for my semi-conduction specific Linux and won't work with your OSV 
based Linux.  That doesn't help anyone (unless they are intentionally trying to 
be incompatible!)  So if we can provide a functional best practice and example 
of how to do this, then when people want to provide (and use) a secondary 
toolchain -- it's available to them.

>>
>> Implementation
>> ---------------------
>>
>> Add an OVERRIDE to specify the alternative toolchain.  Can this be done
>> within the layer by doing something like:
>>
>> OVERRIDE_append = ":toolchain-${TOOLCHAIN}"
>>
>> TOOLCHAIN = "gnu" (or "icc")
>>
>> To activate the toolchain you would use things like:
>>
>> TOOLCHAIN_pn-bash = 'icc'
>>
>> To define the correct behavior for the toolchain, the following would need
>> to be defined (with _toolchain-<toolchain>):
>>
>> TARGET_CPPFLAGS
>> TARGET_CFLAGS
>> TARGET_CXXFLAGS
>> TARGET_LDFLAGS
>> CC
>> CXX
>> F77
>> CPP
>> LD
>> CCLD
>> AR
>> AS
>> RANLIB
>> STRIP
>> OBJCOPY
>> OBJDUMP
>> NM
>> FULL_OPTIMIZATIONS
>> DEBUG_OPTIMIZATION
>>
>> Is anyone aware of any other items that may require additional items?  Will
>> the above actually work?  Using the override of the TOOLCHAIN_… will that
>> actually change the override values or do we get stuck?
>>
>> Comments/suggestions appreciated!
>> --Mark
>>
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core at lists.openembedded.org
>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core