[OE-core] [devtool][PATCH] devtool deploy-target: optionally specify package

[Andre McCurdy]

On Thu, May 31, 2018 at 7:23 PM, Trevor Woerner <twoerner at gmail.com> wrote:
> Hi Andre, thanks for taking an interest!
>
> On Thu, May 31, 2018 at 6:22 PM, Andre McCurdy <armccurdy at gmail.com> wrote:
>>
>> On Thu, May 31, 2018 at 1:06 PM, Trevor Woerner <twoerner at gmail.com>
>> wrote:
>> > Instead of installing an entire recipe's build output (i.e. ${D}), allow
>> > the
>> > user to optionally specify a package from said recipe to be installed
>> > exclusively (i.e. ${PKGDEST}/<package>).
>>
>> I'm not a big devtool user myself, but I'm curious about the
>> situations where you might need this?
>
> Earlier today I was working on recipes for some software which recently
> added the feature to dlopen() one of a couple shared objects based on
> various conditions. But instead of dlopen()'ing the SONAME, they were
> dlopen()'ing the raw .so file. OE's default packaging, of course, added the
> *.so.0 and *.so.0.0.0 files to the ${PN} package, and the raw *.so files to
> the ${PN}-dev package. So when the app tried to run, it failed.

Sounds familiar. I used to maintain this patch for the mesa recipe to
workaround a similar issue in Qt:

  http://lists.openembedded.org/pipermail/openembedded-devel/2015-May/101455.html

I don't know if later versions of Qt still show the problem (and of
course, the workaround in the mesa recipe was never merged...).

> Investigating, I realised there were two things I could do: 1) add the raw
> *.so files to the ${PN} package (with INSANE_SKIP += "dev-so") or 2) fix the
> app. I decided to try #2, with the help of devtool.
>
> I made one small change to the app, "devtool deploy-target"'ed it, and,
> AMAZINGLY, everything worked!! Wow, what luck! ... or not.
>
> Investigating, I discovered that "devtool deploy-target"'s only mode of
> operation is to upload the entire recipe's ${D} directory (i.e.
> ${WORKDIR}/image) to the target! I hope you'll be as surprised by that as am
> I. Considering ${D} includes *.a files, man pages, raw *.so files,
> pkgconfigs, (... thankfully it doesn't also include the -dbg packages) that
> is a lot of stuff to be uploading to the target, a lot of wrong stuff, and
> in my case that hindered my work. Most target filesystems are only 10%
> larger than required to hold their contents, so uploading static libraries,
> man pages, pkgconfigs, etc is not great.
>
> Specifically, these unwanted raw *.so files were hindering my work. My only
> two options were: 1) to use "devtool deploy-target" as-is and then have to
> manually delete all the .so files or 2) come up with a better solution. I
> went with #2 (again).
>
> Many recipes generate just a couple packages: ${PN}, -dbg, -doc, -dev,
> -staticdev, -locale. But some generate a whole bunch. In my case my recipe
> is generating a lot of packages, but I only wanted to investigate one. This
> patch allows a developer to focus on just one package, instead of the entire
> recipe.

All seems reasonable to me. Thanks for taking the time to explain all
the details!

I wonder whether the default behaviour should just be to not deploy
files from the -dev package?

>> Are the going to be problems if users want to deploy more than one
>> package (or switch between a full deploy and a package) and don't
>> deploy and undeploy in the right order?
>
>
> In my patch I added a --package option to "devtool deploy-target" but not to
> "devtool undeploy-target". devtool keeps track of what files were uploaded
> by creating a per-package list on the target itself (in /.devtool). When you
> undeploy a recipe, it simply looks for this recipe file on the target, reads
> the files listed there, and undeploys each of the files in that list.
>
> Therefore, mixing and matching deploys and undeploys amongst recipes don't
> interfere with each other.
>
> I checked at the time, but I'm not 100% sure now, but I believe if one
> deploys two packages from one recipe, the on-target file gets appended. But
> now I'm not 100% sure, so I'll verify this tomorrow (thanks for the
> reminder). As such, under my scheme, you can deploy recipes and packages at
> will, but then only undeploy the whole kit and caboodle in one go; given a
> recipe name to undeploy, whatever it finds in the on-target record will get
> undeployed.
>
> I think any developer will find this reasonable behaviour (but maybe not?).

If that's how it works then it seems reasonable to me.

> In any case, my patch doesn't affect any existing workflows. If a developer
> never notices the new --package option and simply continues to use "devtool
> deploy-target" with just a recipe name, the behaviour will be exactly the
> same. Only when the --package option is used will the new behaviour occur.
>
> I'll verify deploying multiple packages from the same recipe to confirm they
> append the list file. I'll also verify deploying the same package multiple
> times (although my guess is there'll be no difference between this and the
> existing behaviour should one deploy the same recipe multiple times).