[bitbake-devel] Bug with clearing SRCREV cache

[Daniel Lazzari]

>-----Original Message-----
>From: Daniel Lazzari
>Sent: Wednesday, August 03, 2011 11:29 AM
>To: 'Chris Larson'
>Cc: bitbake-devel at lists.openembedded.org
>Subject: RE: [bitbake-devel] Bug with clearing SRCREV cache
>
>>-----Original Message-----
>>From: kergoth at gmail.com [mailto:kergoth at gmail.com] On Behalf Of Chris
>>Larson
>>Sent: Wednesday, August 03, 2011 11:23 AM
>>To: Daniel Lazzari
>>Cc: bitbake-devel at lists.openembedded.org
>>Subject: Re: [bitbake-devel] Bug with clearing SRCREV cache
>>
>>On Wed, Aug 3, 2011 at 11:21 AM, Daniel Lazzari <dlazzari at leapfrog.com>
>>wrote:
>>>>-----Original Message-----
>>>>From: kergoth at gmail.com [mailto:kergoth at gmail.com] On Behalf Of Chris
>>>>Larson
>>>>Sent: Tuesday, August 02, 2011 6:24 PM
>>>>To: Daniel Lazzari
>>>>Cc: bitbake-devel at lists.openembedded.org
>>>>Subject: Re: [bitbake-devel] Bug with clearing SRCREV cache
>>>>
>>>>On Tue, Aug 2, 2011 at 12:06 PM, Daniel Lazzari
>><dlazzari at leapfrog.com>
>>>>wrote:
>>>>> I’m not exactly sure if this is an oe-core or a bitbake bug, so I’m
>>>>trying
>>>>> here first. I noticed the other day that one of our recipes that
>>pulls
>>>>> source from svn and uses SRCREV = “AUTOINC” wasn’t pulling the
>>latest
>>>>> revision even after a cleanall. After a lot of digging, I
>discovered
>>>>that
>>>>> fetcher_init() was clearing the srcrev cache in “tmp-
>angstrom_2010_x
>>“
>>>>> rather than “tmp-angstrom_2010_x-uclibc”. It appears that the
>>>>following line
>>>>> in persist_data.py is the culprit:
>>>>>
>>>>>
>>>>>
>>>>>     cachedir = (bb.data.getVar("PERSISTENT_DIR", d, True) or
>>>>>
>>>>>                 bb.data.getVar("CACHE", d, True))
>>>>>
>>>>>
>>>>>
>>>>> PERSISTENT_DIR doesn’t evaluate to the correct value. Changing it
>to
>>>>just
>>>>> pull from CACHE seems to fix it, but I’m not sure how important
>>>>> PERSISTENT_DIR is or even what it does.
>>>>
>>>>What do you mean by "the correct value" here? PERSISTENT_DIR is
>>>>exactly what it sounds like, the location where persist_data stores
>>>>its database(s), that's it.
>>>>--
>>>>Christopher Larson
>>>>clarson at kergoth dot com
>>>>Founder - BitBake, OpenEmbedded, OpenZaurus
>>>>Maintainer - Tslib
>>>>Senior Software Engineer, Mentor Graphics
>>>
>>> Well, I may just be confused as to what the correct value of
>>PERSISTENT_DIR should be, but I would expect it to be the same value as
>>CACHE "tmp-angstrom_2010_x-uclibc" and not "tmp-angstrom_2010_x". If I
>>do a "bitbake -e linux-libc-headers | grep PERSISTENT_DIR" I get the
>>value I expect, so maybe PERSISTENT_DIR is changing after
>fetcher_init()
>>(where the cache is cleared) but before the first do_fetch() task
>(where
>>the cache is checked for the SRCREV).
>>
>>As khem pointed out, this is due to the use of _append with TMPDIR.
>>Early uses of the variable by bitbake do not have the appended bits
>>applied, later uses do.
>>--
>>Christopher Larson
>>clarson at kergoth dot com
>>Founder - BitBake, OpenEmbedded, OpenZaurus
>>Maintainer - Tslib
>>Senior Software Engineer, Mentor Graphics
>
>Ah, that makes sense. I know that the late binding of variable values in
>bitbake has bitten me before. Still though, this behavior is breaking
>BB_SRCREV_POLICY = "clear" because the wrong SRCREV cache ends up
>getting cleared. This manifests as SRCREV = "AUTOINC" not getting the
>latest revision if a revision number was previously cached, and I'm not
>sure how best to fix it.

Alright, I've finally figured out a fix. I changed defaultsetup.conf in oe-core from TMPDIR_append = "${TCLIBCAPPEND}" to TMPDIR := "${TMPDIR}${TCLIBCAPPEND}"

I'll submit the patch to oe-core. Thanks for everyone's help in tracking this down and fixing it. I really appreciate it!