[oe] meta-oe and yocto-check-layer

Khem Raj raj.khem at gmail.com
Tue Sep 25 17:19:08 UTC 2018 

On Tue, Sep 25, 2018 at 10:17 AM Nicolas Dechesne
<nicolas.dechesne at linaro.org> wrote:
>
> On Tue, Sep 25, 2018 at 7:10 PM Martin Jansa <martin.jansa at gmail.com> wrote:
> >
> > On Tue, Sep 25, 2018 at 03:05:19PM +0200, Nicolas Dechesne wrote:
> > > On Tue, Sep 25, 2018 at 12:29 PM Martin Jansa <martin.jansa at gmail.com> wrote:
> > > >
> > > > On Tue, Sep 25, 2018 at 12:24:31PM +0200, Martin Jansa wrote:
> > > > > > meta-oe: add meta-python in LAYERDEPENDS (needed for protobuf)
> > > > >
> > > > > This causes another circular dependency which we don't want, doesn't it?
> > >
> > > What are the issues with circular dependency? LAYERDEPENDS only lists
> > > each layers own dependencies, which is helpful for integrators to know
> > > which layer they need to pull into bblayers.conf. If all layers from
> > > LAYERDEPENDS are pulled in, then it is expected that each recipe will
> > > build fine.
> >
> > It prevents using these layers separately.
> >
> > Now people can include just meta-oe without meta-python (as long as they
> > fix or mask the python-protobuf dependency from protobuf-ptest - which
> > is a bug not a feature).
> >
> > If they have circular dependency then everybody using meta-oe will be
> > forced to use meta-python as well and then why should we keep them in
> > separate layers? It would be the same as adding all meta-python recipes
> > into meta-oe.
>
> It is not because there are circular dependencies that meta-oe
> requires meta-python, it is because of protobuf , it really depends on
> meta-python... I am tempted to agree with Paul, we should make the
> dependency explicitly optional using PACKAGECONFIG instead of
> requiring users to fix/mask it.
>

packageconfig seems a good approach for now

> >
> > There was similar issue with meta-perl not so long time ago, before with
> > meta-multimedia and meta-networking.. so if we don't fix these issues
> > properly and just add more dependencies from meta-oe, then whole
> > meta-openembedded as a repository will became one layer soon.
>
> agreed. I think it's good to spot these issues and fix them as they come.
>
> >
> > > The only circular dependency that I am aware is with yocto-check-layer
> > > script , and I sent a patch yesterday to fix this issue. With this
> > > patch, yocto-check-layer works fine even when layers have inter
> > > dependencies.
> > >
> > > https://patchwork.openembedded.org/patch/155113/
> > >
> > > >
> > > > Especially if it's caused only by python-protobuf runtime dependency added in:
> > > >
> > > > https://patchwork.openembedded.org/patch/146867/
> > >
> > > yes. this is the culprit.
> > >
> > > >
> > > > > On Tue, Sep 25, 2018 at 11:44 AM Nicolas Dechesne <
> > > > > nicolas.dechesne at linaro.org> wrote:
> > > > >
> > > > > > hi Armin,
> > > > > >
> > > > > > On Tue, Sep 25, 2018 at 8:59 AM Nicolas Dechesne
> > > > > > <nicolas.dechesne at linaro.org> wrote:
> > > > > > >
> > > > > > > On Mon, Sep 24, 2018 at 11:51 PM akuster808 <akuster808 at gmail.com>
> > > > > > wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On 09/24/2018 02:03 PM, Paul Eggleton wrote:
> > > > > > > > > Hi Nicolas,
> > > > > > > > >
> > > > > > > > > On Monday, 24 September 2018 10:05:02 PM NZST Nicolas Dechesne wrote:
> > > > > > > > >> hi Armin, Paul, Richard,
> > > > > > > > >>
> > > > > > > > >> I was looking at getting the compliance report for meta-oe (sumo
> > > > > > > > >> branch), and I have found a few issues.
> > > > > > > > >>
> > > > > > > > >> * in meta-openembedded/sumo, grpc is in meta-oe layer, while it
> > > > > > > > >> depends on meta-networking (c-ares). It was fixes in master, with
> > > > > > > > >> 251878e8b6b9 (grpc: move it from oe to networking layer), so I think
> > > > > > > > >> this fix needs to be backported to sumo as well if we want the YP
> > > > > > 2.0
> > > > > > > > >> compliance script to even work. If agreed, once merged, please let
> > > > > > me
> > > > > > > > >> know so that I can try again to generate a compliance report.
> > > > > > > > > Is it appropriate to make such moves in a stable branch? I wouldn't
> > > > > > have
> > > > > > > > > thought so.
> > > > > > > > >
> > > > > > > >
> > > > > > > > We have to. Per my understanding and why I tried very hard to make
> > > > > > > > meta-openembedded clean ( appears I failed) is that if you want to be
> > > > > > > > Yocto Compliant and include any layer that does not pass this test, you
> > > > > > > > can not become Yocto Compliant.
> > > > > > >
> > > > > > > I believe that we want meta-openembedded to be compliant, and a good
> > > > > > > example in general. I will send a backport your way for this change.
> > > > > >
> > > > > > Running the compliance script on meta-oe turned out to be an
> > > > > > interesting exercise ;)
> > > > > >
> > > > > > I have found several issues, which I have mentioned in a few different
> > > > > > threads, so I will summary here.
> > > > > >
> > > > > > * oe-core: fix the yocto-check-layer for dependency loop
> > > > > > * I have the following local commits in meta-oe:
> > > > > > meta-oe: add meta-python in LAYERDEPENDS (needed for protobuf)
> > > > > > grpc: move it from oe to networking layer
> > > > > > meta-multimedia: fixup LAYERDEPENDS (for dos2unix issue)
> > > > > >
> > > > > > With all changes above, the compliance script finds another issue with
> > > > > > meta-xfce:
> > > > > >
> > > > > > AssertionError: Adding layer meta-xfce changed signatures.
> > > > > > 7 signatures changed, initial differences (first hash before, second
> > > > > > after):
> > > > > >    vim:do_install: 588d445122dccf317f15b0dd852f3888 ->
> > > > > > ec086472d75d663c2fe836b935517810
> > > > > >
> > > > > > This is definitely a violation of one our rule since adding meta-xfce
> > > > > > changed changes vim recipe.
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > Or relax your rules!!!.
> > > > > > > >
> > > > > > > > - armin
> > > > > > > > >> * in order to run the compliance report, i locally added
> > > > > > > > >> networking-layer in meta-oe/conf/layer.conf, and it creates a
> > > > > > > > >> dependency loop since meta-oe <-> meta-networking. I found out that
> > > > > > > > >> yocto-check-layer doesn't like that too much, and brutally fails.
> > > > > > The
> > > > > > > > >> following patch makes yocto-check-layer work again even with
> > > > > > > > >> dependency loop. I am going to do a few more tests and send that
> > > > > > over
> > > > > > > > >> as a patch.
> > > > > > > > >>
> > > > > > > > >> diff --git a/scripts/lib/checklayer/__init__.py
> > > > > > > > >> b/scripts/lib/checklayer/__init__.py
> > > > > > > > >> index 2618416fab..0cc9bf3b6d 100644
> > > > > > > > >> --- a/scripts/lib/checklayer/__init__.py
> > > > > > > > >> +++ b/scripts/lib/checklayer/__init__.py
> > > > > > > > >> @@ -151,11 +151,21 @@ def add_layer_dependencies(bblayersconf,
> > > > > > layer,
> > > > > > > > >> layers, logger):
> > > > > > > > >>          logger.debug('Processing dependencies %s for layer %s.' % \
> > > > > > > > >>                      (depends, layer['name']))
> > > > > > > > >>
> > > > > > > > >> +        # To avoid never ending recursion, we keep track of layers
> > > > > > while
> > > > > > > > >> +        # they are being processed in this 'static' attribute.
> > > > > > > > >> +        if not hasattr(recurse_dependencies, "layers"):
> > > > > > > > >> +            recurse_dependencies.layers = []
> > > > > > > > >> +
> > > > > > > > >>          for depend in depends.split():
> > > > > > > > >>              # core (oe-core) is suppose to be provided
> > > > > > > > >>              if depend == 'core':
> > > > > > > > >>                  continue
> > > > > > > > >>
> > > > > > > > >> +            if depend in recurse_dependencies.layers:
> > > > > > > > >> +                continue
> > > > > > > > >> +
> > > > > > > > >> +            recurse_dependencies.layers.append(depend)
> > > > > > > > >> +
> > > > > > > > >>              layer_depend = _find_layer_depends(depend, layers)
> > > > > > > > >>              if not layer_depend:
> > > > > > > > >>                  logger.error('Layer %s depends on %s and isn\'t
> > > > > > found.' % \
> > > > > > > > > Patch looks reasonable to me FWIW.
> > > > > > > > >
> > > > > > > > > Cheers,
> > > > > > > > > Paul
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > --
> > > > > > _______________________________________________
> > > > > > Openembedded-devel mailing list
> > > > > > Openembedded-devel at lists.openembedded.org
> > > > > > http://lists.openembedded.org/mailman/listinfo/openembedded-devel
> > > > > >
> > > >
> > > > --
> > > > Martin 'JaMa' Jansa     jabber: Martin.Jansa at gmail.com
> >
> > --
> > Martin 'JaMa' Jansa     jabber: Martin.Jansa at gmail.com
> --
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel at lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-devel

More information about the Openembedded-devel mailing list