[oe] Python3 bytecode is stale

[Mike Looijmans]

Most likely the date/time stamps on the files are such that the "py" files 
appear to be newer.

A very simple fix would be to simply delete the ".py" files, they aren't 
really needed anyway so you'll save a lot of space in the process.

To do this compile-time, you could add something like this in a bbappend:
PACKAGES =+ "${PN}-src"
FILES_${PN}-src = "{installdir}/*.py {installdir}/*/*.py"

That'll push the "py" files into another package and prevent them being 
installed by default. Change "{installdir}" into something more appropriate.
Of just delete them in a do_install_append.


On 25-01-19 14:46, Stefan Agner wrote:
> Hi,
> 
> Recently I noticed that Python3 programs got rather slow. Running the
> test application took 12.5 seconds. We are using a read-only file system
> and I immediately suspected the bytecode cache. After mounting the
> rootfs rw, from the second execution on running the test application
> took below 4 seconds.
> 
> By simply executing a Hello World, Python declares all code objects as
> being stale and tries to rewrite them:
> 
> # python3 -v -c 'print("Hello World")'
> import _frozen_importlib # frozen
> import _imp # builtin
> import sys # builtin
> import '_warnings' # <class '_frozen_importlib.BuiltinImporter'>
> import '_thread' # <class '_frozen_importlib.BuiltinImporter'>
> import '_weakref' # <class '_frozen_importlib.BuiltinImporter'>
> import '_frozen_importlib_external' # <class
> '_frozen_importlib.FrozenImporter'>
> import '_io' # <class '_frozen_importlib.BuiltinImporter'>
> import 'marshal' # <class '_frozen_importlib.BuiltinImporter'>
> import 'posix' # <class '_frozen_importlib.BuiltinImporter'>
> import _thread # previously loaded ('_thread')
> import '_thread' # <class '_frozen_importlib.BuiltinImporter'>
> import _weakref # previously loaded ('_weakref')
> import '_weakref' # <class '_frozen_importlib.BuiltinImporter'>
> # installing zipimport hook
> import 'zipimport' # <class '_frozen_importlib.BuiltinImporter'>
> # installed zipimport hook
> # bytecode is stale for 'encodings'
> # code object from /usr/lib/python3.5/encodings/__init__.py
> # could not create
> '/usr/lib/python3.5/encodings/__pycache__/__init__.cpython-35.pyc':
> OSError(30, 'Read-only file system')
> # wrote
> '/usr/lib/python3.5/encodings/__pycache__/__init__.cpython-35.pyc'
> # bytecode is stale for 'codecs'
> # code object from /usr/lib/python3.5/codecs.py
> # could not create
> '/usr/lib/python3.5/__pycache__/codecs.cpython-35.pyc': OSError(30,
> 'Read-only file system')
> # wrote '/usr/lib/python3.5/__pycache__/codecs.cpython-35.pyc'
> import '_codecs' # <class '_frozen_importlib.BuiltinImporter'>
> import 'codecs' # <_frozen_importlib_external.SourceFileLoader object at
> 0xb6b33130>
> # bytecode is stale for 'encodings.aliases'
> # code object from /usr/lib/python3.5/encodings/aliases.py
> # could not create
> '/usr/lib/python3.5/encodings/__pycache__/aliases.cpython-35.pyc':
> OSError(30, 'Read-only file system')
> # wrote
> '/usr/lib/python3.5/encodings/__pycache__/aliases.cpython-35.pyc'
> import 'encodings.aliases' #
> <_frozen_importlib_external.SourceFileLoader object at 0xb6b3d9d0>
> import 'encodings' # <_frozen_importlib_external.SourceFileLoader object
> at 0xb6b154b0>
> # bytecode is stale for 'encodings.ascii'
> # code object from /usr/lib/python3.5/encodings/ascii.py
> # could not create
> '/usr/lib/python3.5/encodings/__pycache__/ascii.cpython-35.pyc':
> OSError(30, 'Read-only file system')
> # wrote '/usr/lib/python3.5/encodings/__pycache__/ascii.cpython-35.pyc'
> import 'encodings.ascii' # <_frozen_importlib_external.SourceFileLoader
> object at 0xb6b3db10>
> import '_signal' # <class '_frozen_importlib.BuiltinImporter'>
> ....
> 
> 
> This is on a Armv7 device. OpenEmbedded layers are current master.
> Python version is 3.5.6. Is this a known issue? Any idea?
> 
> --
> Stefan
>