[oe] Building a kernel with builtin initramfs

[Richard Purdie]

On Fri, 2008-02-15 at 18:29 +0200, Paul Sokolovsky wrote:
> > On Fri, 2008-02-15 at 13:32 +0200, Paul Sokolovsky wrote:
> > This is deliberate. Its so if you have "deb" packaging enabled and
> > then switch to "ipk", everything just works since bitbake realises
> > which stamps are missing and builds the missing parts.
> 
> ;-( That's apparently a case where good rebuild from scratch would
> solve all issues ;-). 

Apart from the time taken. Why waste all the time when we don't need to.
This was a nice solution which fitted into existing frameworks, wasn't
complicated and worked well.

> Or looking at the root of it, making bitbake
> operate on hierarchical tasks would solve this and likely other issues
> we discuss.

What do you mean by hierarchical tasks? It is a hierarchical task
executor already?

> > What I'd prefer to see is a sequence like:
> > 
> > bitbake initramfs-image
> > bitbake initramfs-kernel
> > 
> 
> Ah, you mean sequences of:
> 
> ./configure <lotsa stuff>
> make
> make install <some stuff>

No, I don't mean sequences like that, they're obviously different.

> ? Wait, that holy place is taken already! And OE have been going in
> quite different direction for long time. It automates mind-boggling
> things like building gcc or glibc, but when it comes to automating
> bitbake with bitbake itself, there's suddenly a taboo? ;-)

Yes. It requires a very precise distro setup for this to work. For
example I changed the libc mode I was using in poky the other day and
smashed up a load of my glibc ipks.

You're proposing this as a solution to switching libc modes and who
knows what else and it really isn't an option.

> Running bitbake in recipe is just as visible and understandable as
> running gcc or anything else - nothing is shown to users except for
> some high-level motions, but all actions are recorded in logs. So, to
> understand what's going on, a look at recipe and logs is required in
> either case, and there're lots of "non-recursive" recipes which are
> beyond understanding of mere humans. On the contrary,
> initramfs-image-ipk is pretty easy (except for long lines) and should
> easily teach something new.

No, it breaks things for the user, e.g. "bitbake -g" no longer shows all
the tasks that actually happen. It also won't honour bitbake's threading
instuctions. Oh, and how sure are you that two bitbakes can run on the
same directory at once? (the answer is that they can't and it will break
badly.)

> Other points:
> * It only starts with 2 commands, the list grows with the time
> (exponentially?)

How many of these recursive executions have we ever needed? Can you give
me more examples of cases where we need to do exponential recursive
execution?

> * It's not going to be those easy "bitbake foo + bitbake bar"
> 2-word commands. As was pointed already, building an initramfs requires
> ANGSTROM_MODE (aka LIBC_TYPE aka (but unrecognized) SYSTYPE) and
> IMAGE_FSTYPES set. Further - more.

ANGSTROM_MODE? So this is even angstrom specific and requires special
setup? (it also implies multimachine which is not default)

> * One of the annoying issues is to how to get hands on results of the
> first task in the second one. Currently, for the second task, such
> location is rather unpredictable. I solved this by adding special task
> which just accepts parameter - location where to put stuff. Again,
> special tasks, special parameters. User would either need to know and
> use them, or do it all manually.

We used to have this problem with kernels and images for scripting
purposes which is why we added predictably named symlinks. These kind of
issues are solvable in simple ways with some thought.

I'm actually going to go as far as proposing any recursive bitbake
recipes are removed from OE.dev ASAP for the reasons I've mentioned
above. We *need* to find a better way. I will try and help you find that
and will extend bitbake if needed.

I think the problem is you're effectively trying to end up with
something where you "bitbake angstrom" and it produces a distribution,
uploads it, writes its own documentation and fixes its own bugs. There
is always going to be some effort that comes from a user when directing
bitbake what to do, be it setting the MACHINE, saying what image type
you'd like or what kind of image you want. The problem you're addressing
is one clearly where the user needs to provide some input.

Its ironic that you were arguing *against* my insistence of the
automatic setting of the IMAGE_FSTYPES variable on a per machine
basis ;-).

Cheers,

Richard