<div dir="ltr">I don't think this is right. This unpacks the shallow tarball into ud.clonedir. The fetcher is specifically done the way it is to avoid ud.clonedir ever being shallow so as to avoid having to maintain that state and convert it when necessary.</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Feb 24, 2020 at 3:29 PM Paul Barker <<a href="mailto:pbarker@konsulko.com">pbarker@konsulko.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Mon, 24 Feb 2020 at 22:10, Richard Purdie<br>
<<a href="mailto:richard.purdie@linuxfoundation.org" target="_blank">richard.purdie@linuxfoundation.org</a>> wrote:<br>
><br>
> On Mon, 2020-02-24 at 21:46 +0000, Paul Barker wrote:<br>
> > On Mon, 24 Feb 2020 at 21:32, Richard Purdie<br>
> > <<a href="mailto:richard.purdie@linuxfoundation.org" target="_blank">richard.purdie@linuxfoundation.org</a>> wrote:<br>
> > > On Mon, 2020-02-24 at 20:16 +0000, Paul Barker wrote:<br>
> > > > When a shallow mirror tarball is used to satisfy a gitsm URI it<br>
> > > > needs to<br>
> > > > be unpacked so that the .gitmodules file can be examined.<br>
> > > ><br>
> > > > Signed-off-by: Paul Barker <<a href="mailto:pbarker@konsulko.com" target="_blank">pbarker@konsulko.com</a>><br>
> > > > ---<br>
> > > > lib/bb/fetch2/gitsm.py | 6 ++++++<br>
> > > > 1 file changed, 6 insertions(+)<br>
> > > ><br>
> > > > diff --git a/lib/bb/fetch2/gitsm.py b/lib/bb/fetch2/gitsm.py<br>
> > > > index aa121cbe..98f5dae5 100644<br>
> > > > --- a/lib/bb/fetch2/gitsm.py<br>
> > > > +++ b/lib/bb/fetch2/gitsm.py<br>
> > > > @@ -169,6 +169,12 @@ class GitSM(Git):<br>
> > > > raise<br>
> > > ><br>
> > > > Git.download(self, ud, d)<br>
> > > > +<br>
> > > > + # If we're using a shallow mirror tarball it needs to be<br>
> > > > unpacked so<br>
> > > > + # that we can examine the .gitmodules file<br>
> > > > + if ud.shallow and os.path.exists(ud.fullshallow) and<br>
> > > > Git.need_update(self, ud, d):<br>
> > > > + bb.utils.mkdirhier(ud.clonedir)<br>
> > > > + runfetchcmd("tar -xzf %s" % ud.fullshallow, d,<br>
> > > > workdir=ud.clonedir)<br>
> > > > self.process_submodules(ud, ud.clonedir,<br>
> > > > download_submodule, d)<br>
> > > ><br>
> > > > def unpack(self, ud, destdir, d):<br>
> > ><br>
> > > Does this mean we're missing a test?<br>
> ><br>
> > Possibly. Maybe a two step test with a recipe using a `gitsm://` URL:<br>
> > 1) Fetch the recipe with BB_GENERATE_MIRROR_TARBALLS,<br>
> > BB_GENERATE_SHALLOW_TARBALLS and BB_GIT_SHALLOW set. Copy the mirror<br>
> > tarballs out of the downloads directory to a local mirror directory.<br>
> > Throw away DL_DIR, SSTATE_DIR and TMPDIR.<br>
> > 2) Using the mirror generated in step 1 try to fetch and unpack the<br>
> > recipe.<br>
> ><br>
> > Does that sound like a sensible test? I think it would have to go in<br>
> > oeqa rather than the bitbake tests if we do it that way.<br>
><br>
> It sounds like it might be reasonable, I'm not sure why we can't do<br>
> that in lib/bb/tests/fetch.py though? There are other tests which do<br>
> similar sequences of commands.<br>
><br>
> You'd do it with calls to the fetch and unpack Fetcerh APIs and alter a<br>
> datastore and files in between but I can't see why it would need to be<br>
> in oeqa?<br>
<br>
Ok, I've looked more closely at GitMakeShallowTest in<br>
lib/bb/tests/fetch.py. We could write a new<br>
test_shallow_submodule_mirrors function based on<br>
test_shallow_submodules and test_shallow_mirrors.<br>
<br>
The case I would want to catch is do_unpack failing. I guess we could<br>
check that fetcher.unpack() works as a proxy for that.<br>
-- <br>
_______________________________________________<br>
bitbake-devel mailing list<br>
<a href="mailto:bitbake-devel@lists.openembedded.org" target="_blank">bitbake-devel@lists.openembedded.org</a><br>
<a href="http://lists.openembedded.org/mailman/listinfo/bitbake-devel" rel="noreferrer" target="_blank">http://lists.openembedded.org/mailman/listinfo/bitbake-devel</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr">Christopher Larson<br>kergoth at gmail dot com<br>Founder - BitBake, OpenEmbedded, OpenZaurus<br>Senior Software Engineer, Mentor Graphics</div></div></div></div>