[OE-core] BitBake changes in the Yocto Project 1.5 cycle

[Richard Purdie]

I've been giving some thought to where BitBake needs to go in the future
in order to deliver for its users. It started life as a commandline
utility and its grown a lot since it was first created. I think there
are some key decisions that need to be taken to ensure its future
growth.

The first proposal is that we should change the BitBake server so it
becomes memory resident. This means that the first time you run "bitbake
X", the server loads into memory, then subsequent BitBake commands would
just connect to the server and do things. We'd add in some kind of
timeout of say 15 minutes so that it would gracefully exit.

The reason for doing this is simple, it would allow commands to be much
more responsive rather than having the cache/configuration loading each
time which is where our current overhead is. Obviously it would detect
changes to things like MACHINE setting, local.conf and re-parse as
normal in those cases. The intent would be to speed up the interaction
with the system so you don't have the annoying delays/lag.

This would then lead into the second proposal which is to add better
support for new commands. This would be along the lines of some of
Chris' work with bb, the bitbake-env script and so on but with the added
advantage of being able to connect to the server for info. It would
allow us to add query commands for things like the available
PACKAGECONFIG options/settings much more easily than the current
infrastructure. There are some issues about whether these would belong
as part of BitBake or OE and we'd need to figure that out.

The third proposal is to do away with the BitBake wrapper script in
OE-Core for pseudo. Doing this within BitBake is going to be hard and
require some rework but ultimately, I think it will be worth it from a
performance perspective and also stops some of the user confusion about
BitBake running twice (two sets of log files and so on).

These changes should also be in keeping with the expanded UI work and
options such as WebHob and allowing remote use of multiple UIs connected
to servers.

Feedback/comments/suggestions etc./ welcome as always.

I've cc'd the OE-Core list but further discussion should be taken to
bitbake-devel.

Cheers,

Richard