[OE-core] [PATCH 0/2] recipetool: add python buildsystem support

[Paul Eggleton]

Hi Chris,

On Thursday 08 January 2015 11:24:08 Christopher Larson wrote:
> - Handles distutils & setuptools.
> - Supports pulling metadata from PKG-INFO, .egg-info, & setup.py (via two
>   different mechanisms).
> - Doesn't handle python 3 yet.
> - Optional dependencies via `extra_requires` are not currently
>   handled.
> - Implements scanning for dependencies by using the python ast to
>   determine what modules/packages are being imported by the python
>   code in question
> 
> See https://gist.github.com/kergoth/6cca6caa4e9b22a21f75 for the projects
> I used to test this against, and
> https://gist.github.com/kergoth/7fea9494e3cf60b382ff which is the result of
> running recipetool create against every recipe that inherits distutils or
> setuptools in my build environment.
> 
> https://gist.github.com/kergoth/7ad4b5102b2a5fe3a956 has my current TODO for
> this feature.

This works pretty well, thanks!

I tried this on python-mako and got this:

------------- snip --------------
...
# WARNING: the following rdepends are from setuptools install_requires. These
# upstream names may not correspond exactly to bitbake package names.
RDEPENDS_${PN} += "markupsafe"

# WARNING: the following rdepends are determined through basic analysis of the
# python sources, and might not be 100% accurate. It is your responsibility to
# verify that the values are complete and correct.
RDEPENDS_${PN} += "python-core python-textutils"

# WARNING: We were unable to map the following python package/module
# dependencies to the bitbake packages which include them:
#    mako.lookup
#    mako.template
#    sys
------------- snip --------------

* Shouldn't it know that sys is provided by python-core (maybe this has to be 
hardcoded)? 

* Possibly also that "mako" is a module provided by this recipe?

* Our existing recipe has "python-threading python-netclient python-html" for 
RDEPENDS_${PN}, I don't know if all of these are correct but it does at least 
try to import threading.

* Should we just assume by convention that package names should start with 
"python-" and thus make that prefix on all items brought in from setuptools 
install_requires?

* I think "It is your responsibility ..." is a little harsh for the dependency 
situation - you'll soon find out at runtime if the value is insufficient. I used 
this wording for LICENSE because if you don't check it yourself you won't find 
out if it's incorrect.

Earlier you mentioned it would be nice if the buildsystem handler could 
replace the existing value of LICENSE - I checked and it already can; you'll 
find the existing value in lines_before. It's all a little bit clunky API-wise 
but it is at least possible.

I also noticed when looking at this that my code wasn't replacing the version 
with ${PV} in S (but was in SRC_URI). I'll send a patch to fix that. The way 
the code works at the moment, I only replace it in SRC_URI if you've specified 
a file name containing the version and then we can find that same version in the 
SRC_URI value (I'm not sure there is any other practical way).

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre