[OE-core] ValueError: insecure string pickle with latest bitbake

Martin Jansa martin.jansa at gmail.com
Sun Feb 23 09:06:40 UTC 2014 

On Sat, Feb 22, 2014 at 01:07:36PM +0100, Martin Jansa wrote:
> Hi,
> 
> last couple of days many of my jenkins builds failed with exception like
> this:
> 
> ERROR: An uncaught exception occured in runqueue, please see the failure below:
> ERROR: Running idle function
> Traceback (most recent call last):
>   File "/home/jenkins/oe/shr-core-branches/shr-core/bitbake/lib/bb/cooker.py", line 1182, in buildTargetsIdle(server=<ProcessServer(ProcessServer-2, started)>, rq=<bb.runqueue.RunQueue instance at 0x16a95d40>, abort=False):
>                  try:
>     >                retval = rq.execute_runqueue()
>                  except runqueue.TaskFailure as exc:
>   File "/home/jenkins/oe/shr-core-branches/shr-core/bitbake/lib/bb/runqueue.py", line 1093, in RunQueue.execute_runqueue():
>                  try:
>     >                self.teardown_workers()
>                  except:
>   File "/home/jenkins/oe/shr-core-branches/shr-core/bitbake/lib/bb/runqueue.py", line 925, in RunQueue.teardown_workers():
>              self.workerpipe = None
>     >        self._teardown_worker(self.fakeworker, self.fakeworkerpipe)
>              self.fakeworker = None
>   File "/home/jenkins/oe/shr-core-branches/shr-core/bitbake/lib/bb/runqueue.py", line 906, in RunQueue._teardown_worker(worker=<subprocess.Popen object at 0xcb4e690>, workerpipe=<bb.runqueue.runQueuePipe instance at 0x11c17710>):
>              while worker.returncode is None:
>     >            workerpipe.read()
>                  worker.poll()
>   File "/home/jenkins/oe/shr-core-branches/shr-core/bitbake/lib/bb/runqueue.py", line 2095, in runQueuePipe.read():
>                  while index != -1 and self.queue.startswith("<event>"):
>     >                event = pickle.loads(self.queue[7:index])
>                      bb.event.fire_from_worker(event, self.d)
> ValueError: insecure string pickle
> 
> and it repeats the same exception very quickly (generated 3GB log in ~ hour)
> 
> It doesn't happen always, it seems to be triggered in different places in build.
> 
> Anyone else seeing this as well?
> 
> I'll add some debug output around it and wait for it to happen again.

I got one example now (not sure if this is the only one):

Because it shows runrpmdeps and I'm the only one so far seeing this, I guess it's caused by:
http://lists.openembedded.org/pipermail/openembedded-core/2014-February/089632.html
I'll try to revert that one from my builds

NOTE: recipe binutils-2.24-r0: task do_configure: Started
ERROR: failed load pickle 'ccopy_reg
_reconstructor
p1
(clogging
LogRecord
p2
c__builtin__
object
p3
NtRp4
(dp5
S'taskpid'
p6
I10922
sS'threadName'
p7
S'MainThread'
p8
sS'name'
p9
S'BitBake'
p10
sS'thread'
p11
I139823574271744
sS'created'
p12
F1393145646.5582349
sS'process'
p13
I11633
sS'processName'
p14
S'PoolWorker-11'
p15
sS'args'
p16
(tsS'module'
p17
S'__init__'
p18
sS'filename'
p19
'__init__.py'
p20
sS'levelno'
p21
I10
sS'exc_text'
p22
NsS'pathname'
p23
S'/OE/build/shr-core/bitbake/lib/bb/__init__.py'
p24
sS'lineno'
p25
I58
sS'msg'
p26
S"runrpmdeps: /OE/build/shr-core/tmp-eglibc/sysroots/x86_64-linux/usr/lib/rpm/bin/rpmdeps-oecore --macros /OE/build/shr-core/tmp-eglibc/sysroots/x86_64-linux/usr/lib/rpm/macros --define '_rpmfc_magic_path /OE/build/shr-core/tmp-eglibc/sysroots/x86_64-linux/usr/share/misc/magic.mgc' --rpmpopt /OE/build/shr-core/tmp-eglibc/sysroots/x86_64-linux/usr/lib/rpm/rpmpopt /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/debug/forward_list /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/debug/multimap.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/debug/multiset.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/debug/safe_local_iterator.tcc /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/debug/safe_sequence.tcc /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/debug/safe_unordered_container.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/debug/string /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/debug/array /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/debug/safe_sequence.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/backward/strstream /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/backward/hash_fun.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/backward/hash_set /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/backward/auto_ptr.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/backward/hashtable.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/backward/binders.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/backward/backward_warning.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/backward/hash_map /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/exp_integral.tcc /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/float.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/limits.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/hypergeometric.tcc /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/cfenv /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/random.tcc /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/fenv.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/gamma.tcc /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/wchar.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/math.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/functional_hash.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/riemann_zeta.tcc /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/inttypes.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/cctype /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/cstdio /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/shared_ptr.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/cinttypes /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/unordered_set.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/ccomplex /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/poly_hermite.tcc /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/random.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/unordered_set /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/ctgmath /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/random /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/complex /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/utility /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/functional /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/modified_bessel_func.tcc /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/cstdbool /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/type_traits /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/cwctype /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/wctype.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/package
s-split/libstdc++-dev/usr/include/c++/tr1/stdlib.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/poly_laguerre.tcc /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/cfloat /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/special_function_util.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/cstdarg /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/beta_function.tcc /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/unordered_map.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/unordered_map /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/memory /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/regex /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/hashtable.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/stdio.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/climits /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/complex.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/legendre_function.tcc /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/bessel_function.tcc /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/hashtable_policy.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/cstdlib /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/tuple /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/cwchar /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/cmath /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/ctype.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/ell_integral.tcc /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/tgmath.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/stdarg.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/stdbool.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/cstdint /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/ctime /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/stdint.h /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr1/array /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr2/ratio /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr2/bool_set.tcc /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr2/dynamic_bitset /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr2/type_traits /OE/build/shr-core/tmp-eglibc/work/core2-64-oe-linux/gcc-runtime/4.8.2-r0/packages-split/libstdc++-dev/usr/include/c++/tr2/bool_set /OE/build/shr-core/tmp-eglibc/work/c<event>ccopy_reg
p1
(clogging
LogRecord
p2
c__builtin__
object
p3
NtRp4
(dp5
S'taskpid'
p6
I10922
sS'threadName'
p7
S'MainThread'
p8
sS'name'
p9
S'BitBake'
p10
sS'thread'
p11
I139823574271744
sS'created'
p12
F1393145646.558748
sS'process'
p13
I11662
sS'processName'
p14
S'PoolWorker-13'
p15
sS'args'
p16
(tsS'module'
p17
S'__init__'
p18
sS'filename'
p19
S'__init__.py'
p20
sS'levelno'
p21
I10
sS'exc_text'
p22
NsS'pathname'
p23
S'/OE/build/shr-core/bitbake/lib/bb/__init__.py'
p24
sS'lineno'
p25
I58
sS'msg'
p26

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa at gmail.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20140223/d620489f/attachment-0002.sig>

More information about the Openembedded-core mailing list