[OE-core] Never ending stream of bitbake exceptions when the builder runs out of disk space

[Martin Jansa]

This issue exists for very long time.

I know that when the builder runs out of disk space there are multiple
things which might go wrong (I've seen bad archives on premirrors, bad
sstate archives caused by this), so this issue isn't the main problem, but
still would be nice to fail faster.

In last build which was running for some 9 hours, it was first building for
maybe 2 hours before it run out of disk space and this morning there is
50MB log just from bitbake output stored on the jenkins master. Repeating
following message very quickly

# grep -c "Errno 28" consoleText.txt
42986

ERROR: Running command [['world'], 'build']
Traceback (most recent call last):
  File "/home/jenkins/oe/world/shr-core/bitbake/lib/bb/event.py", line 211,
in fire(event=<bb.event.HeartbeatEvent object at 0x7fcfed3e96a0>,
d=<bb.data_smart.DataSmart object at 0x7fd00330b198>):

    >    fire_class_handlers(event, d)
         if worker_fire:
  File "/home/jenkins/oe/world/shr-core/bitbake/lib/bb/event.py", line 134,
in fire_class_handlers(event=<bb.event.HeartbeatEvent object at
0x7fcfed3e96a0>, d=<bb.data_smart.DataSmart object at 0x7fd00330b198>):
                         continue
    >            execute_handler(name, handler, event, d)

  File "/home/jenkins/oe/world/shr-core/bitbake/lib/bb/event.py", line 106,
in execute_handler(name='runqueue_stats', handler=<function runqueue_stats
at 0x7fd0020c6158>, event=<bb.event.HeartbeatEvent object at
0x7fcfed3e96a0>, d=<bb.data_smart.DataSmart object at 0x7fd00330b198>):
         try:
    >        ret = handler(event)
         except (bb.parse.SkipRecipe, bb.BBHandledException):
  File
"/home/jenkins/oe/world/shr-core/openembedded-core/meta/classes/buildstats.bbclass",
line 212, in runqueue_stats(e=<bb.event.HeartbeatEvent object at
0x7fcfed3e96a0>):
             done = isinstance(e, bb.event.BuildCompleted)
    >        system_stats.sample(e, force=done)
             if done:
  File
"/home/jenkins/oe/world/shr-core/openembedded-core/meta/lib/buildstats.py",
line 148, in SystemStats.sample(event=<bb.event.HeartbeatEvent object at
0x7fcfed3e96a0>, force=False):
                                      data +
    >                                 b'\n')
                 self.last_proc = now
OSError: [Errno 28] No space left on device

It would be better to exit completely when something as bad as Errno 28
happens.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20170615/410547ef/attachment-0002.html>