[OE-core] bitbake eating more than 6GB ram while parsing recipes

[Khem Raj]

On Sun, Oct 23, 2011 at 12:57 AM, Martin Jansa <martin.jansa at gmail.com> wrote:
> Hi,
>
> I know this was reported a while ago and I had this issue for few days
> with 4 GB RAM on my workstation it was hard to do anything when bitbake
> was parsing and today I've "resolved" it by removing cache.

yes I have exact same problem. As you add more machines to same tmpdir
build. It keeps becoming s bigger pig. And so far I have been removing
cache dir too since s parse from scratch is way faster in this case.

I think we have to profile bitbake and find the memory wastage even after
parse right now it keeps like 600M through out the build process which
could be reduced too.

>
> first I've tried to remove
> #PARALLEL_MAKE="-j3"
> #BB_NUMBER_THREADS = "2"
> from my local.conf and it didn't help I still had 6 bitbake processes
> 4 eating ~ 1GB ad 2 with ~ 512MB. So I've removed tmp/cache except
> bb_persist_data.sqlite3:
>
> OE om-gta02 at shr ~/shr-core $ du -hs tmp/cache/*
> 35M     tmp/cache/bb_codeparser.dat
> 0       tmp/cache/bb_codeparser.dat.lock.13293
> 0       tmp/cache/bb_codeparser.dat.lock.13294
> 0       tmp/cache/bb_codeparser.dat.lock.13295
> 0       tmp/cache/bb_codeparser.dat.lock.15708
> 0       tmp/cache/bb_codeparser.dat.lock.16878
> 0       tmp/cache/bb_codeparser.dat.lock.16879
> 0       tmp/cache/bb_codeparser.dat.lock.16880
> 0       tmp/cache/bb_codeparser.dat.lock.25605
> 68K     tmp/cache/bb_persist_data.sqlite3
> 87M     tmp/cache/default-eglibc
> OE om-gta02 at shr ~/shr-core $ rm -rf tmp/cache/bb_codeparser.dat*
>
> OE om-gta02 at shr ~/shr-core $ du -hs tmp/cache/default-eglibc/*
> 13M     tmp/cache/default-eglibc/nokia900
> 13M     tmp/cache/default-eglibc/om-gta02
> 13M     tmp/cache/default-eglibc/palmpre
> 13M     tmp/cache/default-eglibc/qemuarm
> 13M     tmp/cache/default-eglibc/qemux86
> 13M     tmp/cache/default-eglibc/qemux86-64
> 13M     tmp/cache/default-eglibc/spitz
> OE om-gta02 at shr ~/shr-core $ rm -rf tmp/cache/default-eglibc/
>
> and now it's sane again and memory used is < 2GB.
>
> Regards,
> --
> Martin 'JaMa' Jansa     jabber: Martin.Jansa at gmail.com
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core at lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
>