[bitbake-devel] Bitbake for native x86 project?

[Chris Morgan]

On Sun, Oct 13, 2013 at 12:53 PM, Richard Purdie
<richard.purdie at linuxfoundation.org> wrote:
> On Fri, 2013-10-11 at 09:22 -0400, Chris Morgan wrote:
>> I'm working on a project that generates several libraries and
>> applications. Today we build and run exclusively on x86 Linux desktop
>> machines and use cmake for the build system. In the future we plan to
>> add cross-compiling of these same libraries and applications for an
>> embedded arm platform, Yocto based. We would like to maintain both
>> types of build, for x86 and for arm.
>>
>> I've seen guides on how to use Yocto (or other bitbake based
>> distributions) to generate x86 images but these appear to involve
>> building the entire x86 toolchain, kernel, all libraries, creating a
>> virtual root and building a root filesystem image that one would run
>> under an emulator.
>
> This isn't entirely accurate. You can choose just to build a given
> library for example (bitbake libfoo). All its default dependencies are
> also built though, which means in most cases, a cross toolchain would be
> triggered. There is no need to build a kernel or a complete rootfs.
>
>> In my case I'd like to use bitbake to build libraries and applications
>> and create a virtual root, but not create a kernel, toolchain and
>> other system provided libraries. Then, when we decide to transition
>> over to Yocto I'd like to leverage the recipes by plugging them into
>> the Yocto system somehow (I'm still starting to understand the
>> process).
>
> To try and clarify, we have roughly three types of target recipes:
>
> a) target system
>
> These are for the target system defined by MACHINE and are cross
> compiled by a cross toolchain against a cross compiled libc. target can
> be different to the build architecture. They are packaged and managed by
> a package manager.
>
> b) nativesdk
>
> These are standalone binaries designed to run on SDKMACHINE. They are
> cross compiled and link against their own libc which is designed to run
> in some location defined at install time. We use these to generate
> toolchain binaries and tools that can run on pretty much an Linux
> system, regardless of its kernel, libc or system layout. They are
> packaged and managed by a package manager.
>
> c) native
>
> These are compiled to run on the build system. They work very much like
> you describe except they are not packaged at all.
>
> (I'm ignoring -cross and -crosssdk since these are basically toolchain
> bootstrap details).
>
>> I haven't seen any guides or information on how to use bitbake for
>> native desktop applications. I feel like its certainly possible but it
>> appears to be quite a bit of work just to proof of concept things.
>>
>> Is there anything inherently wrong with what I'm proposing here with
>> bitbake? Does it make sense that it would save time to develop the
>> recipes once and use bitbake for the desktop like this?
>
> Its certainly possible, its just that nobody has done it. Your options
> would be to either reconfigure the native recipes to be packaged, or to
> add in a configuration which shortcircuits the cross compiler and adds
> in some native magic to specific machine targets.
>
> Whilst adding packaging to native sounds attractive, the reality is that
> its not easy and doesn't really make sense technically. Adding in those
> short-circuits I mention by comparison shouldn't be hard and if I were
> doing this, its the way I'd go.
>
> Cheers,
>
> Richard
>


Ahh interesting. I was thinking that this kind of a short-circuit
approach might make sense. Its good to hear that the concept isn't so
crazy although I've literally found no information about anyone doing
anything of the sort on the web.

I considered using the full blown x86/x86-64 targets but our build
time right now using cmake and native tools is ~30 seconds with only a
handful of MB of data to download.

Would using a sstate cache + a custom image target be able to get down
to that kind of time? Today each developer is building on their
desktop and from my own testing with angstrom it looks like the builds
take nearly an hour and generate 30+GB of data. This is almost
entirely why I've been hesitant to try to use the built-in x86/x86-64
support. I love the idea of ensuring that everything matches exactly
on each developer machine but the overhead is pretty crazy.

Chris