[bitbake-devel] [meta-ti] trouble during parsing

[Andreas Müller]

On Mon, Feb 25, 2013 at 10:46 AM, Javier Martinez Canillas
<javier at dowhile0.org> wrote:
> On Mon, Feb 25, 2013 at 10:32 AM, Andreas Müller
> <schnitzeltony at googlemail.com> wrote:
>> On Mon, Feb 25, 2013 at 3:28 AM, Flanagan, Elizabeth
>> <elizabeth.flanagan at intel.com> wrote:
>>> On Sun, Feb 24, 2013 at 6:12 PM, Flanagan, Elizabeth
>>> <elizabeth.flanagan at intel.com> wrote:
>>>> On Wed, Feb 20, 2013 at 6:31 AM, Chris Larson <clarson at kergoth.com> wrote:
>>>>>
>>>>> On Wed, Feb 20, 2013 at 5:54 AM, Andreas Müller
>>>>> <schnitzeltony at googlemail.com> wrote:
>>>>>>
>>>>>> with current bitbake master I get
>>>>>>
>>>>>> ERROR: Command execution failed: Traceback (most recent call
>>>>>> last):#########################################################
>>>>>>                               | ETA:  00:00:17
>>>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/command.py", line
>>>>>> 92, in runAsyncCommand
>>>>>>     self.cooker.updateCache()
>>>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>>>>>> 1330, in updateCache
>>>>>>     if not self.parser.parse_next():
>>>>>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>>>>>> 1703, in parse_next
>>>>>>     self.virtuals += len(result)
>>>>>> TypeError: object of type 'ExpansionError' has no len()
>>>>>
>>>>>
>>>>> Hmm, looks like it's returning the exceptions rather than raising them, for
>>>>> some reason, but that doesn't make much sense — the pool code always raises
>>>>> any exceptions from its imap iterator's next() method.
>>>>> --
>>>>> Christopher Larson
>>>>>
>>>>
>>>> I'm hitting this as well with
>>>> poky:dde7a481354d5b0539762109bdfaaba6f85f879b, meta-ti and pandaboard
>>>> as my machine.
>>>>
>>>
>>> Ach, should have updated my email. Seems reverting
>>> 0a99563a4ea270594fd9a61da46f9387fb79dc66 cleared up the issue.
>>>
>>> -b
>>>
>> I have two machines (Fedora 14  / Fedora 15)  both reverted
>> 0a99563a4ea270594fd9a61da46f9387fb79dc66 and both worked fine during
>> weekend.
>>
>> Just a guess: Last week when arago git was down I saw errors during
>> parsing for a recipe I never used. This was with a revision of bitbake
>> a bit earlier than HEAD - that caused me to update. After sending a
>> message to meta-ti I could work during weekend. Now it seems arago is
>> down again:
>>
>> $ git clone git://arago-project.org/git/projects/u-boot-keystone.git
>> Cloning into u-boot-keystone...
>> fatal: read error: Connection reset by peer
>>
>> and bitbake -DD gives
>>
>> DEBUG: Fetcher accessed the network with the command git ls-remote
>> git://arago-project.org/git/projects/u-boot-keystone.git
>> DEV.MCSDK-03.00.00.07
>> DEBUG: Running export SSH_AUTH_SOCK="/tmp/keyring-Pob6mG/ssh"; export
>> PATH="/home/andreas/data/oe-core/sources/openembedded-core/scripts:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/bin/armv7a-vfp-neon-angstrom-linux-gnueabi:/home/andreas/tmp/oe-core-eglibc/sysroots/overo/usr/bin/crossscripts:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/sbin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/usr/bin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux/sbin:/home/andreas/tmp/oe-core-eglibc/sysroots/x86_64-linux//bin:/home/andreas/oe-core/sources/bitbake/bin:/home/andreas/data/OpenEmbedded/bitbake/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:./:/home/andreas/bin:/sbin:/usr/sbin";
>> export HOME="/home/andreas"; git ls-remote
>> git://arago-project.org/git/projects/u-boot-keystone.git
>> DEV.MCSDK-03.00.00.07
>> DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
>> argument 'preinst' is not a string literal
>> DEBUG: while parsing sstate_install, in call of bb.build.exec_func,
>> argument 'postinst' is not a string literal
>>
>> ...
>>
>> DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
>> argument 'preinst' is not a string literal
>> DEBUG: while parsing sstate_install, in call of bb.build.exec_func,
>> argument 'postinst' is not a string literal
>> DEBUG: while parsing do_clean, in call of bb.build.exec_func, argument
>> 'f' is not a string literal
>> DEBUG: while parsing sstate_installpkg, in call of bb.build.exec_func,
>> argument 'preinst' is not a string literal
>> ERROR: Command execution failed: Traceback (most recent call last):
>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/command.py", line
>> 92, in runAsyncCommand
>>     self.cooker.updateCache()
>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>> 1329, in updateCache
>>     if not self.parser.parse_next():
>>   File "/home/andreas/oe-core/sources/bitbake/lib/bb/cooker.py", line
>> 1702, in parse_next
>>     self.virtuals += len(result)
>> TypeError: object of type 'ExpansionError' has no len()
>>
>>
>> In the recipe u-boot_2013.01.bb I find SRCREV = "DEV.MCSDK-03.00.00.07"
>>
>> which is not a commit ID and causes a git ls-remote during parse
>>
>> Is it possible that this error occurs when a git SRC_URI server closes
>> connection unexpected?
>>
>> Andreas
>
> Hi Andreas,
>
> I have the same issue and your guess is correct. The problem is that
> Arago servers are down so "git ls-remote" is failing and this makes
> the BitBake task to fail as well.
>
> Now about the why is been used tag names instead of just commits IDs
> so a "git ls-remote" call is needed, this is because some git trees in
> arago are been rebased overwriting the commit IDs.
>
> You may take a look to Robert P. J. Day email "[meta-ti] objections to
> replacing git SRCREV tag names with actual      hash IDs?" [1] to get a
> better understanding of this.
>
> Best regards,
> javier
>
> [1]: http://www.mail-archive.com/meta-ti@yoctoproject.org/msg01304.html

One additional note: With current bitbake there are configurations
which cause a full parse on every build [1]. Is it possible that
arrago-git dies for the flood of  git ls-remote?

Andreas

[1] http://lists.linuxtogo.org/pipermail/bitbake-devel/2013-February/004178.html