[oe] meta-oe and yocto-check-layer

Tue Sep 25 13:05:19 UTC 2018

On Tue, Sep 25, 2018 at 12:29 PM Martin Jansa <martin.jansa at gmail.com> wrote:
>
> On Tue, Sep 25, 2018 at 12:24:31PM +0200, Martin Jansa wrote:
> > > meta-oe: add meta-python in LAYERDEPENDS (needed for protobuf)
> >
> > This causes another circular dependency which we don't want, doesn't it?

What are the issues with circular dependency? LAYERDEPENDS only lists
each layers own dependencies, which is helpful for integrators to know
which layer they need to pull into bblayers.conf. If all layers from
LAYERDEPENDS are pulled in, then it is expected that each recipe will
build fine.

The only circular dependency that I am aware is with yocto-check-layer
script , and I sent a patch yesterday to fix this issue. With this
patch, yocto-check-layer works fine even when layers have inter
dependencies.

https://patchwork.openembedded.org/patch/155113/

>
> Especially if it's caused only by python-protobuf runtime dependency added in:
>
> https://patchwork.openembedded.org/patch/146867/

yes. this is the culprit.

>
> > On Tue, Sep 25, 2018 at 11:44 AM Nicolas Dechesne <
> > nicolas.dechesne at linaro.org> wrote:
> >
> > > hi Armin,
> > >
> > > On Tue, Sep 25, 2018 at 8:59 AM Nicolas Dechesne
> > > <nicolas.dechesne at linaro.org> wrote:
> > > >
> > > > On Mon, Sep 24, 2018 at 11:51 PM akuster808 <akuster808 at gmail.com>
> > > wrote:
> > > > >
> > > > >
> > > > >
> > > > > On 09/24/2018 02:03 PM, Paul Eggleton wrote:
> > > > > > Hi Nicolas,
> > > > > >
> > > > > > On Monday, 24 September 2018 10:05:02 PM NZST Nicolas Dechesne wrote:
> > > > > >> hi Armin, Paul, Richard,
> > > > > >>
> > > > > >> I was looking at getting the compliance report for meta-oe (sumo
> > > > > >> branch), and I have found a few issues.
> > > > > >>
> > > > > >> * in meta-openembedded/sumo, grpc is in meta-oe layer, while it
> > > > > >> depends on meta-networking (c-ares). It was fixes in master, with
> > > > > >> 251878e8b6b9 (grpc: move it from oe to networking layer), so I think
> > > > > >> this fix needs to be backported to sumo as well if we want the YP
> > > 2.0
> > > > > >> compliance script to even work. If agreed, once merged, please let
> > > me
> > > > > >> know so that I can try again to generate a compliance report.
> > > > > > Is it appropriate to make such moves in a stable branch? I wouldn't
> > > have
> > > > > > thought so.
> > > > > >
> > > > >
> > > > > We have to. Per my understanding and why I tried very hard to make
> > > > > meta-openembedded clean ( appears I failed) is that if you want to be
> > > > > Yocto Compliant and include any layer that does not pass this test, you
> > > > > can not become Yocto Compliant.
> > > >
> > > > I believe that we want meta-openembedded to be compliant, and a good
> > > > example in general. I will send a backport your way for this change.
> > >
> > > Running the compliance script on meta-oe turned out to be an
> > > interesting exercise ;)
> > >
> > > I have found several issues, which I have mentioned in a few different
> > > threads, so I will summary here.
> > >
> > > * oe-core: fix the yocto-check-layer for dependency loop
> > > * I have the following local commits in meta-oe:
> > > meta-oe: add meta-python in LAYERDEPENDS (needed for protobuf)
> > > grpc: move it from oe to networking layer
> > > meta-multimedia: fixup LAYERDEPENDS (for dos2unix issue)
> > >
> > > With all changes above, the compliance script finds another issue with
> > > meta-xfce:
> > >
> > > AssertionError: Adding layer meta-xfce changed signatures.
> > > 7 signatures changed, initial differences (first hash before, second
> > > after):
> > >    vim:do_install: 588d445122dccf317f15b0dd852f3888 ->
> > > ec086472d75d663c2fe836b935517810
> > >
> > > This is definitely a violation of one our rule since adding meta-xfce
> > > changed changes vim recipe.
> > >
> > > >
> > > > >
> > > > > Or relax your rules!!!.
> > > > >
> > > > > - armin
> > > > > >> * in order to run the compliance report, i locally added
> > > > > >> networking-layer in meta-oe/conf/layer.conf, and it creates a
> > > > > >> dependency loop since meta-oe <-> meta-networking. I found out that
> > > > > >> yocto-check-layer doesn't like that too much, and brutally fails.
> > > The
> > > > > >> following patch makes yocto-check-layer work again even with
> > > > > >> dependency loop. I am going to do a few more tests and send that
> > > over
> > > > > >> as a patch.
> > > > > >>
> > > > > >> diff --git a/scripts/lib/checklayer/__init__.py
> > > > > >> b/scripts/lib/checklayer/__init__.py
> > > > > >> index 2618416fab..0cc9bf3b6d 100644
> > > > > >> --- a/scripts/lib/checklayer/__init__.py
> > > > > >> +++ b/scripts/lib/checklayer/__init__.py
> > > > > >> @@ -151,11 +151,21 @@ def add_layer_dependencies(bblayersconf,
> > > layer,
> > > > > >> layers, logger):
> > > > > >>          logger.debug('Processing dependencies %s for layer %s.' % \
> > > > > >>                      (depends, layer['name']))
> > > > > >>
> > > > > >> +        # To avoid never ending recursion, we keep track of layers
> > > while
> > > > > >> +        # they are being processed in this 'static' attribute.
> > > > > >> +        if not hasattr(recurse_dependencies, "layers"):
> > > > > >> +            recurse_dependencies.layers = []
> > > > > >> +
> > > > > >>          for depend in depends.split():
> > > > > >>              # core (oe-core) is suppose to be provided
> > > > > >>              if depend == 'core':
> > > > > >>                  continue
> > > > > >>
> > > > > >> +            if depend in recurse_dependencies.layers:
> > > > > >> +                continue
> > > > > >> +
> > > > > >> +            recurse_dependencies.layers.append(depend)
> > > > > >> +
> > > > > >>              layer_depend = _find_layer_depends(depend, layers)
> > > > > >>              if not layer_depend:
> > > > > >>                  logger.error('Layer %s depends on %s and isn\'t
> > > found.' % \
> > > > > > Patch looks reasonable to me FWIW.
> > > > > >
> > > > > > Cheers,
> > > > > > Paul
> > > > > >
> > > > > >
> > > > >
> > > --
> > > _______________________________________________
> > > Openembedded-devel mailing list
> > > Openembedded-devel at lists.openembedded.org
> > > http://lists.openembedded.org/mailman/listinfo/openembedded-devel
> > >
>
> --
> Martin 'JaMa' Jansa     jabber: Martin.Jansa at gmail.com