[Openembedded-architecture] OEDEM - Setup program

[Mark Hatle]

This is an attempt to answer questions on what the hell this setup program is
all about.  For now I'm going to refer to it as a openembedded-chef... lacking a
better name.

As I've said before, I have certain requirements inside of the company I work
for before I can release the code.  So the best I can do is present a demo or
give more information until that time.


Purpose:

Based on prior OEDAM and OEDEM meetings it was stressed that a component was
needed to help users generate a distribution builder project.  This new
component has a single purpose; using the layer index, setup a new distribution
builder project.  In other words, gather the proper ingredients necessary to
build a distribution.


What it does:

The program will go out to a layer index, specified in a settings file,
defaulting to 'layers.openembedded.org'.  It will query the index on a specific
branch.  Using the information, and the command line arguments the user presents
it, it will construct a new git repository, populate the repository with a repo
xml 'default.xml' file that contains all of the components necessary to build
that configuration, construct README and variable 'sample' files, and finally
setup symlinks (via repo) to things like oe-init-build-env, etc.

The end result will be a specific distribution builder (project) that contains
all of the layers and their dependencies the user has requested.

Workflow:

The following is the expected workflow for a user.  There are two variants.  The
primary workflow downloads everything from the internet.  The secondary workflow
is intended to mirror the content locally so other people can pull from the mirror.

Primary:

$ mkdir my-openembedded-project
$ cd my-openembedded-project
$ git clone --branch morty git://git.openembedded.org/openembedded-chef
$ ./openembedded-chef/setup.sh --machine qemux86 --layer meta-networking
<system runs>

results in:

A git repository created in the CWD.  Inside of this new git repository are all
of the layers necessary, symlinks to make it work, etc.

$ ls
bitbake
config
default.xml
layers
meta
oe-init-build-env
oe-init-build-env-memres
README
scripts

$ ls layers/
meta-openembedded
openembedded-core

$ cat default.xml
<manifest>
    <remote  name="base" fetch="git://git.openembedded.org"/>
    <default revision="morty" remote="base" sync-j="4"/>
    <project name="bitbake" remote="base"
path="layers/openembedded-core/bitbake" revision="1.32">
        <linkfile src="." dest="bitbake"/>
    </project>
    <project name="meta-openembedded" remote="base"
path="layers/meta-openembedded" revision="morty">
    </project>
    <project name="openembedded-core" remote="base"
path="layers/openembedded-core" revision="morty">
        <linkfile src="oe-init-build-env" dest="oe-init-build-env"/>
        <linkfile src="oe-init-build-env-memres" dest="oe-init-build-env-memres"/>
        <linkfile src="meta" dest="meta"/>
        <linkfile src="scripts" dest="scripts"/>
    </project>
</manifest>


The system also uses it's knowledge of the layers it has downloaded to create
custom sample files:

bblayers.conf.sample  conf-notes.txt  local.conf.sample

These files are automatically processed by the existing oe-init-build-env script.


secondary (mirror):

$ mkdir my-openembedded-mirror
$ cd my-openembedded-mirror
$ git clone --branch morty git://git.openembedded.org/openembedded-chef
$ ./openembedded-chef/setup.sh --machine qemux86 --layer meta-networking --mirror



Usage information:

usage: setup.py [-h] [--list-distros] [--list-machines] [--list-layers]
                [--list-recipes] [--distros distro]
                [--machines MACHINE [MACHINE ...]]
                [--layers LAYER [LAYER ...]] [--recipes RECIPE [RECIPE ...]]
                [--all-layers] [--no-recommend] [-v] [-rj REPO_JOBS]
                [--base-url URL] [--base-branch BRANCH] [--mirror]

setup.py: Application to fetch & setup an OpenEmbedded project.

optional arguments:
  -h, --help            show this help message and exit
  --list-distros        List all available distro values.
  --list-machines       List all available machine values.
  --list-layers         List all available layers.
  --list-recipes        List all available recipes.
  --distros distro      Select layer(s) based on required distribution and set
                        the default DISTRO= value (default nodistro)
  --machines MACHINE [MACHINE ...]
                        Select layer(s) based on required machine(s) and set
                        the default MACHINE= value (default qemux86-64)
  --layers LAYER [LAYER ...]
                        Select layer(s) to include in the project and add to
                        the default bblayers.conf
  --recipes RECIPE [RECIPE ...]
                        Select layers(s) based on recipe(s)
  --all-layers          Select all available layers
  --no-recommend        Disable recommended layers during layer resolution
  -v, --verbose         Set the verbosity to debug
  -rj REPO_JOBS, --repo-jobs REPO_JOBS
                        Sets repo project to fetch simultaneously (default 4)
  --base-url URL        URL to fetch from (default git://git.openembedded.org)
  --base-branch BRANCH  Core branch identifier (default morty)
  --mirror              Do not construct a project, instead construct a mirror
                        for other projects

When will code be available?

The code is still in active development.  We have every intention to release the
code between now and mid December -- pending my required internal review.

What is the intention after the code is available?

Per the OEDEM meeting, everyone in the room (no objection consensus) said they
believed that it should be part of the OpenEmbedded project and hosted on
git.openembedded.org.

I want to stress here.  I do NOT intend to wait for the code to be 'done',
'polished', etc.  We will publish the sources as soon as I am able.  This was
presented that if it was useful, we would be happy to publish it as part of the
OpenEmbedded project.  Alternatively as part of the Yocto Project.  If nobody
other then use thought it was useful it will end up on our own github site.