[Openembedded-architecture] Heterogeneous System Proposal

Mon Dec 2 23:27:56 UTC 2019

Hey guys,

On Mon, Dec 2, 2019 at 3:24 PM Mark Hatle <mark.hatle at kernel.crashing.org>
wrote:

>
>
> On 12/2/19 4:09 PM, Richard Purdie wrote:
> > On Mon, 2019-12-02 at 21:58 +0000, Richard Purdie wrote:
> >> On Mon, 2019-12-02 at 15:37 -0600, Mark Hatle wrote:
> >>> In all of the above examples, the user has manually configured the
> >>> multiconfig within their project.  There is a simple way to move
> >>> that
> >>> configuration to a layer, simply place it in a conf/multiconfig
> >>> directory within that layer.
> >>>
> >>> This ability suggests to be that there should be a standard way to
> >>> specify a layer above that of machine, which defines the overall
> >>> characteristics of the system.
> >>>
> >>> I'm proposing calling this new layer type as a system layer and
> >>> it's
> >>> configuration variable "SYSTEM".  It will be used instead of
> >>> MACHINE
> >>> when there is no single machine to quantify the contents of the
> >>> produced system image.  When implementing a system, we do not want
> >>> to
> >>> make major changes to any other components.  Due to the existing
> >>> implementation requiring MACHINE and certain TUNE parameters, this
> >>> will require us to provide a special MACHINE value that can be used
> >>> for a heterogeneous system.  I suggest we create a new 'nomachine'
> >>> system that only defines/uses an equivalent noarch style
> >>> tune.  This
> >>> will instruct the system that this configuration can be used to
> >>> noarch software and create images (including with wic), but it is
> >>> not
> >>> able to compile specific applications.  Each of these applications
> >>> or
> >>> images must come from a defined MACHINE.
> >>
> >> I'm not sure I 'buy' this explanation and I think a "nomachine"
> >> target will be horrible in practice. In your example below, why can't
> >> mysystem.conf just set MACHINE = "zcu_cortex-a72" and the other bits
> >> of linux.conf and BBMULTICONFIG = "bootloader fpga" ?
> >
> > In fact, if you create a conf/machine/system-zcu_cortex-a72.conf and
> > have it set something like:
> >
> > BBMULTICONFIG = "bootloader fpga"
> > MACHINE = "zcu_cortex-a72"
> > require conf/machine/${MACHINE}.conf
> >
> > I think what I/you describe may even work today without a new variable
> > and simply a new convention of a special set of new machine name
> > targets in layers...
>

So I have tried something similar to this, and while it does work, due to
some parsing issue (I still have not figured out exactly what)
the multiconfig dependencies between multiconfigs are not being taken into
account, which defeats the whole purpose.

While I understand that introducing a new variable might create some
confusion, at the same time I do believe that it would be
better if we do it that way, and in some way separate workflows when we are
building a "full" heterogeneous system vs a single
architecture system, the difference lies in the fact that we dont set what
the "master" or core MACHINE is, they are all equal.

I have been working on the implementation of this proposal and here is what
I currently have for which of course I am open to change.

Like I said, I had issues with defining a placeholder machine for the
multiconfig dependencies, so I did go with defining

SYSTEM = "dummy-system"

on local.conf (also not a huge fan of the name since to me its a bit too
generic,but as RP said, the name is probably the least of our problems)

then
include conf/system/${SYSTEM}.conf

and I have the following set up on a layer:

meta-heterogeneous
  conf/layer.conf
  conf/system/dummy-system.conf
  conf/multiconfig/dummy-arm64.conf
  conf/multiconfig/dummy-x86-64.conf
  recipes-heterogeneous/heterogeneous-example_1.0.bb

Explanation:
layer.conf  - normal layer.conf, nothing out of the ordinary
dummy-system.conf - Defines BBMULTICONFIG as dummy-arm64 and dummy-x86-64,
AND defines the multiconfig dependencies between them
dummy-*64.conf - Define each of the multiconfigs, in them there are only 3
variables, DISTRO, MACHINE (qemuarm64 and qemux86-64 respectively), and
TMPDIR which is simpy defined as tmp-${MACHINE}
heterogeneous-example_1.0.bb : This was a placeholder recipe, since to my
knowledge bitbake required a target, and it simply contained a line that
set up a multiconfig dependency that would allow:

$ bitbake heterogeneous-example

to start a buld for core-image-minimal (for example) for all archs in the
SYSTEM.

Now that I read RP's email about BBTARGETS I believe this can be improved
and at the very least delete the example recipe, let me know what yo guys
think

Cheers,

Alejandro

>
> Yes, SYSTEM could be implemented as something like:
>
> MACHINE ??= ${@'system-${SYSTEM}' if d.getVar('SYSTEM')}
>
> (I'm not really sure if that will work on not...)
>
> The key though, I want a way to specify a system that by itself can't build
> binaries -- but can only include binaries from other architectures.  (It
> -CAN-
> however build noarch/all [text file] components to be included in the
> image.)
>
> So you end up with:
>
> system-zcu.conf
> zcu-cortex-a72.conf
> zcu-r5.conf
> zcu-microblaze.conf
>
> In the end the users should be able to build each of the components
> independently WITHOUT ever using system, if they choose the 'old' way of
> combining parts and pieces post build outside of the build system.
>
> If they want a pre-defined multiconfig build, then they could use the
> system-zcu
> to select all of the defaults for them.
>
> --Mark
>
> > Cheers,
> >
> > Richard
> >
> _______________________________________________
> Openembedded-architecture mailing list
> Openembedded-architecture at lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-architecture
>

-- 
*M.S. Alejandro Enedino Herna**ndez Samaniego*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-architecture/attachments/20191202/b5bbef4a/attachment.html>