[oe] Bitbake Architecture, Roadmap, Maintainers and the future

[Richard Purdie]

On Fri, 2010-12-31 at 15:24 +0100, Esben Haabendal wrote:
> While I do not and cannot be considered a maintainer in any form, I do
> have a special use for BitBake.  I use an (almost) upstream BitBake as
> part of the OE-lite project, and as such has rather strong investment in
> BitBake.
> 
> I do not use BitBake as a command, but as a Python module, as OE-lite
> has a different dependency and runqueue/cooker model.  Because of this I
> hope to find some time improve the separation between the parser, data
> and cooker in BitBake, but I will surely notice and have to do something
> about it if BitBake is regressed with regards to this.

I know we've talked about your use of bitbake and I'm aware of it at
least.

The split between the code in general in bitbake is in general ever
improving in my opinion. Chris has some some really great work on
bitbake recently, particularly making it more pythonic. Some of the
changes just before the holidays, particularly the UI changes on the
cooker to UI relationship make me uncomfortable, particularly if they
mean the xmlrpc client/server code no longer works. My concern is that
they cross the boundary into what I'd class as regressing an API
boundary, particularly one that is only in the early stages of being
properly established and that we very much need. I don't know if this is
an API you use or not but its the kind of thing we need to be careful
about. FWIW, its independent of the parallel parsing code and other
changes which is partly why I'm so concerned it was merged as is :(.

> As for merging with sstate, OE-lite uses a completely different staging
> model (simple per-recipe package based), and I would be very unhappy if
> merging sstate into BitBake will make it impossible for OE-lite to use
> upstream BitBake.  I haven't looked enough into the sstate
> implementation with regards to this yet though.

To put your mind at rest, there is no "policy" enforced by bitbake on
the way staging works in sstate. You might find the checksums useful and
you might be able to use them in your code but its all opt in. I'm
therefore not worried about this breaking OE-lite. If there are
problems, please just let us/me know and we'll see how we can fix that
but since everything is backwards compatible, it shouldn't be an issue.

> Most of all, I would like to stress that there actually are other users
> of BitBake than OE and Poky. I hope this is considered a good thing seen
> from a BitBake perspective, and not a disturbance.

Its what the bitbake developers have always wanted/intended. It doesn't
mean bitbake won't change as the need for functionality appears but we
will do our best to keep things backwards compatible within reason
(although replaced functionality will still get phased out over time if
nobody needs it).

I would suggest it would be time well spent for you to find a way to be
able to use a standard upstream bitbake rather than requiring any patch
set as it will then be easier to understand what functionality and APIs
are being used and what impact changes to bitbake have.

> If possible, perhaps we could have a small BitBake meeting at FOSDEM,
> trying to coordinate the way forward?

I think I will be at FOSDEM and as always, am happy to talk :). I don't
think Chris is likely to be there based on previous FOSDEMs but I could
be wrong.

Cheers,

Richard