[bitbake-devel] [PATCH] fetch/gitsm: avoid live submodule fetching during unpack()

[Matt Hoosier]

On Mon, May 14, 2018 at 8:18 AM, Peter Kjellerstedt <
peter.kjellerstedt at axis.com> wrote:

> > -----Original Message-----
> > From: bitbake-devel-bounces at lists.openembedded.org [mailto:bitbake-
> > devel-bounces at lists.openembedded.org] On Behalf Of Matt Hoosier
> > Sent: den 11 maj 2018 21:27
> > To: bitbake-devel at lists.openembedded.org
> > Subject: [bitbake-devel] [PATCH] fetch/gitsm: avoid live submodule
> > fetching during unpack()
> >
> > Although the submodules' histories have been fetched during the
> > do_fetch() phase, the mechanics used to clone the workdir copy
> > of the repo haven't been transferring the actual .git/modules
> > directory from the repo fetched into downloads/ during the
> > fetch task.
> >
> > Fix that, and for good measure also explicitly tell Git to avoid
> > hitting the network during do_unpack() of the submodules.
> >
> > Rev3 changes:
> >
> > - Really fix the shallow-clone issue claimed to be addressed in rev2.
> >
> > Rev2 changes:
> >
> > - Fixed a failure when unpacking a shallow cloned mirror repository.
>
> The Rev2 and Rev3 changes information should go below the --- below
> as it should not be part of the commit message as it relates to the
> patch submission process rather than the actual change. If any of it
> is relevant for the commit message, then the commit message should
> be changed as well.
>

Thanks; done. Resubmitted (as v4) to the list.


>
> > [YOCTO #12739]
> >
> > Signed-off-by: Matt Hoosier <matt.hoosier at gmail.com>
> > ---
> >  lib/bb/fetch2/gitsm.py | 28 +++++++++++++++++++++++++++-
> >  1 file changed, 27 insertions(+), 1 deletion(-)
> >
> > diff --git a/lib/bb/fetch2/gitsm.py b/lib/bb/fetch2/gitsm.py
> > index 0aff1008..3969f2f5 100644
> > --- a/lib/bb/fetch2/gitsm.py
> > +++ b/lib/bb/fetch2/gitsm.py
> > @@ -132,4 +132,30 @@ class GitSM(Git):
> >
> >          if self.uses_submodules(ud, d, ud.destdir):
> >              runfetchcmd(ud.basecmd + " checkout " +
> ud.revisions[ud.names[0]], d, workdir=ud.destdir)
> > -            runfetchcmd(ud.basecmd + " submodule update --init
> --recursive", d, workdir=ud.destdir)
> > +
> > +            updateflags = ""
> > +
> > +            # Copy over the submodules' fetched histories too.
> > +            if ud.bareclone:
> > +                repo_conf = ud.destdir
> > +            else:
> > +                repo_conf = os.path.join(ud.destdir, '.git')
> > +
> > +            if os.path.exists(ud.clonedir):
> > +                # This is not a copy unpacked from a shallow mirror
> clone. So
> > +                # the manual intervention to populate the .git/modules
> done
> > +                # in clone_shallow_local() won't have been done yet.
> > +                runfetchcmd("cp -pr %s %s" % (os.path.join(ud.clonedir,
> 'modules'), repo_conf), d)
> > +                updateflags += " --no-fetch"
> > +            elif os.path.exists(os.path.join(repo_conf, 'modules')):
> > +                # Unpacked from a shallow mirror clone. Manual
> population of
> > +                # .git/modules is already done.
> > +                updateflags += " --no-fetch"
> > +            else:
> > +                # This isn't fatal; git-submodule will just fetch it
> > +                # during do_unpack().
> > +                bb.error("submodule history not retrieved during
> do_fetch()")
> > +
> > +            # Careful not to hit the network during unpacking; all
> history should already
> > +            # be fetched.
> > +            runfetchcmd(ud.basecmd + " submodule update --init
> --recursive %s" % updateflags, d, workdir=ud.destdir)
> > --
> > 2.13.6
>
> //Peter
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/bitbake-devel/attachments/20180514/b2011a7c/attachment-0002.html>