[bitbake-devel] [OE-core] BitBake changes in the Yocto Project 1.5 cycle

[Richard Purdie]

On Tue, 2013-04-23 at 10:13 +0200, Peter Kjellerstedt wrote:
> > -----Original Message-----
> > From: openembedded-core-bounces at lists.openembedded.org
> > [mailto:openembedded-core-bounces at lists.openembedded.org] On Behalf Of
> > Richard Purdie
> > Sent: den 22 april 2013 16:16
> > To: bitbake-devel
> > Cc: openembedded-core
> > Subject: [OE-core] BitBake changes in the Yocto Project 1.5 cycle
> > 
> > I've been giving some thought to where BitBake needs to go in the
> > future in order to deliver for its users. It started life as a 
> > commandline utility and its grown a lot since it was first created. 
> > I think there are some key decisions that need to be taken to 
> > ensure its future growth.
> > 
> > The first proposal is that we should change the BitBake server so 
> > it becomes memory resident. This means that the first time you run
> > "bitbake X", the server loads into memory, then subsequent BitBake 
> > commands would just connect to the server and do things. We'd add 
> > in some kind of timeout of say 15 minutes so that it would 
> > gracefully exit.
> > 
> > The reason for doing this is simple, it would allow commands to be 
> > much more responsive rather than having the cache/configuration 
> > loading each time which is where our current overhead is. Obviously 
> > it would detect changes to things like MACHINE setting, local.conf 
> > and re-parse as normal in those cases. The intent would be to speed 
> > up the interaction with the system so you don't have the annoying 
> > delays/lag.
> 
> How will a bitbake command know which server to connect to in case 
> of multiple concurrent builds of different products on the same 
> computer? Or do you intend for one server to keep track of all builds?

There would continue to be one server per build. I tried to keep away
from specifics of the exact implementation since the answer right now is
"don't know". I do have ideas, for example we may do this through some
kind of file in the build directory like the bitbake.lock file and/or
some kind of variable behaving like DISPLAY. I don't really want to
reinvent the wheel though and am open to other ideas. Dbus falls down
with the "remote" part of what we want to do since its typically only
been used locally and whilst people had ideas about remote parts for it,
I don't think they were ever developed extensively.

> > These changes should also be in keeping with the expanded UI work 
> > and options such as WebHob and allowing remote use of multiple UIs
> > connected to servers.
> 
> How should a remote UI know which server to connect to (if there 
> are multiple ones)?

Right now there are commandline options to bitbake for this. This may
become some kind of DISPLAY equivalent environment variable, not sure
yet.

Cheers,

Richard