[OE-core] is sstate-cache really deterministic together with shlibs providers?

[Martin Jansa]

On Fri, Jun 07, 2013 at 09:14:09PM +0100, Phil Blundell wrote:
> On Fri, 2013-06-07 at 21:20 +0200, Martin Jansa wrote:
> > Imagine this process
> > 
> > foo.bb DEPENDS on libabc.bb to provide libabc.1.so
> > 
> > evil recipe bar.bb installs some binary crap in /opt/crap and because it's
> > picky about libabc version it bundles own compy of libabc.so.1 and
> > installs it to /opt/crap/lib/libabc.so.1
> > 
> > At the time of first build nobody notices libabc.1.so and foo, bar and
> > libabc are happily populated to sysroot.
> > 
> > foo.bb is rebuilt because of some unrelated change, but this times it
> > uses bar as shlibs provider for libabc.so.1
> 
> It sounds like the real bug here is that the shlibdeps code doesn't
> issue a diagnostic when it encounters this kind of ambiguous situation.
> It ought to be perfectly capable of figuring out that there are two
> copies of libabc.so.1 at large which belong to different packages and,
> absent additional guidance, it has no reliable way of knowing which one
> of those is going to satisfy the DT_NEEDED in any random binary.
> 
> If shlibdeps stopped with an error in that situation, which arguably it
> ought to, then the erroneous dependency would never get into sstate and
> the scenario that you describe would not be able to happen.

I like both suggestions from you and from Richard, there is another
slightly related issue with
https://bugzilla.yoctoproject.org/show_bug.cgi?id=4102

It's probably possible to create test case where I create sstate package
with libabc.so.1 provided by MACHINE_ARCH foo and then try to reuse this
sstate but with libabc.so.1 provided by TUNE_PKGARCH foo (e.g. with
different version) shlibs code will still find it provider by
MACHINE_ARCH foo and there isn't way to clean foo from incremental build
(only manually).

I'll create bug report and try to sum it there:
1) RP1: We should perhaps limit shlibs to the default linker search paths?
2) RP2+PB: Warning/Error if two things provide the same shlibs
   - probably better to error out when bar.bb is being built and tries
     to provide the same libabc libabc.bb, it can be in different order
     as bar does not depends on libabc, so next time it can show error
     when building libabc, but IMHO better then showing error when foo 
     finds out 2 possible libabc providers
3) Restrict possible providers to stuff included in dependencies - 
   I don't know how difficult it will be to implement this (to see all
   dependencies including transitive), but IMHO this one is most 
   important and can also solve #4102, most tricky part would be to 
   prevent shlib providers silently disappearing from RDEPENDS - 
   when something doesn't declare dependency explicitly, but it was
   always there when do_package was executed - now missing providers
   are silently ignored IIRC. Buildhistory can probably help a lot to
   find missing explicit deps for shlibs providers when we restrict it.

   This will also ensure that when shlibs provider is changed, the
   package which is automagically RDEPENDing on it gets rebuilt too.

https://bugzilla.yoctoproject.org/show_bug.cgi?id=4628
-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa at gmail.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20130607/0ddddae7/attachment-0002.sig>