[OE-core] [PATCH v2] package: Record PE and PR values for shlib dependencies

[Böszörményi Zoltán]

2018-01-14 12:27 keltezéssel, Richard Purdie írta:
> On Fri, 2018-01-12 at 14:46 +0100, Böszörményi Zoltán wrote:
>> When downgrading a package or using a substitute with lower version,
>> the way to do it is adding or increasing PE and there may be other
>> reasons to set PE.
>>
>> But it doesn't directly help dependant packages because the shlib
>> records only contain PV.
>>
>> Let's add the PE value into the shlib records for packages where
>> it's set.
>>
>> The in-memory variables storing the versions now use the PE:PV
>> notation but the on-disk files must use something else because
>> the : character is already used as field delimiter in the
>> package.list
>> files storing the shlib records. Use # instead in the files,
>> so the file format doesn't change. Conversion occurs on reading
>> and writing the package.list files.
> 
> Can you explain a bit more about why/how this causes a problem?
> 
> PE is needed to maintain consistency of package feeds but the shlibs
> code is primarily used for build time dependency analysis to figure out
> which packages need to depend upon what and ensure runtime dependencies
> are met. I'm not sure that should care about PE.

See my mail about describing the use case at
http://lists.openembedded.org/pipermail/openembedded-devel/2018-January/116291.html

Practically it can prevent proper dist-ugprade because
library package dependencies can be installed later
than binaries requiring them. If this happens, running
any binary may crash in the pkg_postinst scriptlet.

My use case was the eglibc_linaro-2.19 -> glibc_2.24 transition
but any upgrade where the package already has a PE value set
may be in the same situation. E.g. glib-2.0 1:2.48.2 in Morty vs
1:2.52.3 in Rocko.

A bbappend file can also modify PV. In case PE is missing from
the shlib deps, you can even end up with the same situation
in the same distro with extra layers.

> For PR, again its primarily for package feeds so that updates are
> detected and I'm not sure why the shlibs code should need to care about
> it.

Besides PE, it is also better to always ship libraries and
binaries from the same build, i.e. with the same PV-PR value.

Such an example is e.g. util-linux with util-linux-mount and
libmount1.

The above described situation may occur if a patch is backported
that adds a new library function which is also used at the same
time from binaries in the same patch. All without increasing
the actual package version. This may be a rare case but I am sure
it is not without an example in Yocto. I am not sure if a recipe
review enforces changing PR in this case, though, but it should.

> Perhaps if you explain more about the issues you're having which this
> fixes I'll better understand the problem/need.

I think I just did.

Automatically relying on the full [PE:]PV[-PR] (note the optional
parts which are also done automatically in the patch) adds even
better consistency in the repo and upgrades can be done in the
correct order.

Upgrades should even work if I just cherry-pick some upgrades
but not all, i.e. "opkg upgrade <something-using-glib-2.0>"
should pull the libglib-2.0_0 version it was built with which is
not happening now because the library dependency is only (>= 2.52.3)
in Rocko but 1:2.48.2 from Morty satisfies it.

OPKG 0.3.x has a dist-upgrade mode and Rocko also introduced dnf,
so I assume at some point a proper distro upgrade will get supported.

Best regards,
Zoltán Böszörményi