[OE-core] [PATCH 1/4] sstate: allow specifying indirect dependencies to exclude from sysroot

[Richard Purdie]

On Thu, 2018-08-09 at 10:00 +0100, André Draszik wrote:
> I should maybe explain my ultimate goal here:
> 
> meta-java is completely self-hosted, which is nice. But to be able to
> compile OpenJDK v8, various other Java compilers need to be compiled
> first.
> This of course means that all those different Java
> compilers/VMs/runtimes
> and tools needed to compile them end up in each Java recipe's
> sysroot.
> 
> While in theory this is fine, and all of them can be installed in
> parallel,
> as everything resides below subdirectories in /usr/lib/jvm/... and
> usr/bin/java normally is a symlink to the default chosen via update-
> alternatives, this doesn't work in yocto for -native recipes.
> 
> In particular this is annoying, because 'usr/bin/java' in the recipe
> sysroot
> is a *very* old version of java, with no possibility of control.
> 
> This has caused me a bit of headache, and if it was possible to avoid
> all
> the recursive OpenJDK-8 build time dependencies then:
> * usr/bin/java could disappear (for -native)
> * tools inherited from the OpenJDK-8 build can be avoided during
> other
>   recipe builds
> 
> In other words, as a follow-up to this series here I have a local
> meta-java
> change to meta-java's layer.conf:
> 
> SSTATE_EXCLUDEDEPS_SYSROOT += "\
>     openjdk-8-native->(ant-native|attr-native|coreutils-
> native|icedtea7-native|libxslt-native|make-native|openssl-native|zip-
> native|unzip-native) \
> "
> 
> This really is a similar situation to *-initial treatment in
> sstate.bbclass,
> but for Java / OpenJDK.
> 
> 
> So rather than having to add knowledge about Java / OpenJDK into
> sstate.bbclass, I need a way to inject that instead.
> 
> 
> Given the above, would you be happy to consider a series where the
> infrastructure is put in place, but without the flex-native and
> bison-native 
> changes? I.e. patches 1 through 3, but not patch 4 of this series?
> (With
> your previous review comments addressed, of course).
> 
> 
> > Basically binutils-cross is bust with errors like:
> > 
> > > x86_64-poky-linux-ar: error while loading shared libraries:
> > > libfl.so.2:
> > > cannot open shared object file: No such file or directory
> > > make[5]: *** [libaudio_resampler_sse.la] Error 127
> > 
> > There are a ton of other failures, I've not waded through them but
> > I'm
> > guessing there are similar issues.
> 
> I am not sure why I didn't see these in my builds before submitting,
> but now
> I do, too.
> 
> It looks like the cross binutils-native (or more specifically cross
> -ar)
> dynamically links against libflex-native. So the last patch in this
> series
> is of course responsible for that.
> 
> Comparing to my debian, ar isn't dynamically linked against libflex,
> I
> wonder why yocto does it, or why debian doesn't.
> 
> In any case my builds succeed now by dropping the flex-native from
> patch 4
> of this series. Alternative would be to switch to static linking for
> libflex. But see above.

The above is good info thanks. I'm not against the series, I originally
thought you wanted to optimise out flex-native and the test results
showed it didn't work so I was waiting on your response to that as the
series didn't build as it stood. I'd mentioned there were problems but
couldn't remember the specifics, you pushed me to "prove" it, I gave
you the errors.

The flex-native change doesn't work due to the binutils dependency and
may not work in general for anything linking to libfl. It would be
interesting to investigate why we have that linkage and if we can avoid
it. (Note that bison-native may work as it doesn't use a library iirc
so we could split that patch into two?).

I continue to be worried about the regular expression usage. I believe
modern python does some caching behind the scenes which helps but tight
loops with regexps in have caused us pain in the past, particularly
recompiling the expression each time and I worry about new API which
ties us to this. Is it possible to use glob?

I'm not against the patch series, we just need to drop the flex-native
change (maybe adding a special case for binutils if we can figure out
the linking) and figure out if we can use a simpler mechanism than
regexps or at least stop me worrying about performance. I'd need to
spend more time than I have right now to figure out how to properly
benchmark that function but as I said we have been bitten before :(.

Cheers,

Richard