[OE-core] [PATCH] bitbake: gitsm: download submodules

[Freihofer, Adrian]

-----Original Message-----
From: Paul Barker <pbarker at konsulko.com>
To: "Freihofer, Adrian" <adrian.freihofer at siemens.com>
Cc: openembedded-core at lists.openembedded.org <
openembedded-core at lists.openembedded.org>
Subject: Re: [OE-core] [PATCH] bitbake: gitsm: download submodules
Date: Wed, 04 Mar 2020 09:59:44 +0000

On Wed, 4 Mar 2020 08:12:27 +0000
"Freihofer, Adrian" <adrian.freihofer at siemens.com> wrote:

> The unpack function failed because the submodules were not
> downloaded.
> Calling download before unpack for each submodule solves this issue.
> 
> Signed-off-by: Adrian Freihofer <adrian.freihofer at siemens.com>
> ---
>  bitbake/lib/bb/fetch2/gitsm.py | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/bitbake/lib/bb/fetch2/gitsm.py
> b/bitbake/lib/bb/fetch2/gitsm.py
> index c622771d21..3715e9824f 100644
> --- a/bitbake/lib/bb/fetch2/gitsm.py
> +++ b/bitbake/lib/bb/fetch2/gitsm.py
> @@ -184,6 +184,7 @@ class GitSM(Git):
>  
>              try:
>                  newfetch = Fetch([url], d, cache=False)
> +                newfetch.download()
>                  newfetch.unpack(root=os.path.dirname(os.path.join(re
> po
> _conf, 'modules', module)))
>              except Exception as e:
>                  logger.error('gitsm: submodule unpack failed: %s %s'
> %
> (type(e).__name__, str(e)))

You shouldn't be trying to download submodules in the do_unpack step.
If
they're missing at this stage it probably indicates a fetch issue.

Basically true, but the information about which submodules need to be
downloaded is not available before the top level repo has been
unpacked. I guess the solution must be something like:

fetch top-level
unpack top-level
foreach submodule:
  fetch submodule
  unpack submodule

What's the exact error that you're seeing?

Don't remeber exactly. But when I debugged the flow, it became obvious,
that the submodules are not downloaded.

This could be related to the issue I saw when the fetcher uses git
shallow
tarballs from a mirror - if that's the case I'm planning to get that
fixed
this weekend.

Yes, the problem occurs with git shallows. The patch solves it with and
without shallows for us.

Also, your email client seems to have chewed the patch up. It's best to
send
patches using `git send-email` only.

Sorry, we switched to a new e-mail solution. I have to adapt my setup.