[oe] BitBake parallel parsing testing

[Chris Larson]

On Tue, Nov 23, 2010 at 11:26 AM, Koen Kooi <k.kooi at student.utwente.nl> wrote:
> On 23-11-10 15:29, Chris Larson wrote:
>
>> Thanks for your time, and do let me know what you think, either of the
>> code, performance in general, or the new progress bar (shows an ETA),
>
> The heuristic for the progress needs some work, it currently seems to
> take in account all the parsed recipes instead of the last N recipes.

That's what our progress display always did, it's just more obvious now.

> Try ctrl-c'ing it at 75% and restarting, you'll see the ETA taking weird
> values. In my case it stayed on "10 seconds" for more than a minute.

The issue is one I've discussed with foerster a fair bit already.
When you're fully cached, the majority of the time isn't in the
parsing, it's in the loading of the cache file from disk, which isn't
part of our progress at all right now.  Either we stall for seconds
before the progress bar comes up, or we don't appear to stall, but the
ETA is a bit off.

Long term the intent is to properly integrate cache loading into this
progress bar, or into its own -- see
https://github.com/foerster/bitbake/commit/320b114d92181b3f944776b7e8c8db4b868ee3f1
for foerster's initial work on this.

After thinking about what you mentioned about the recipe count being
parsed, and given the fact that cached recipes aren't "parsed",
they're loaded from disk and then added to our CacheData, perhaps we
should change the parsing progress display to truly only show the
recipes being *parsed*, which is what you implied, and add a "Loading
cache..." message before the progress bar comes up, so we don't appear
to stall, then add a proper progress bar for the cache file loading as
a second step.

Alternatively, we change the progress display to truly indicate what's
being done.. if we consider all recipes, it's not *parsing* that we're
doing, its .. perhaps.. processing?

*ponders*
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics