[bitbake-devel] [PATCH] bb.fetch.git: add a way to avoid git protocol, and force http or https mirrors

[Andre McCurdy]

On Tue, Jul 25, 2017 at 7:43 AM, Mark Hatle <mark.hatle at windriver.com> wrote:
> On 7/24/17 4:04 PM, Andre McCurdy wrote:
>> On Tue, May 31, 2016 at 6:50 AM, Richard Purdie
>> <richard.purdie at linuxfoundation.org> wrote:
>>> On Tue, 2016-05-31 at 15:18 +0200, fabien.proriolpatch at kazoe.org wrote:
>>>> From: Fabien Proriol <fabien.proriol at jdsu.com>
>>>>
>>>> This patch add the possibility to bitbake to avoid git protocol to
>>>> fetch sources.
>>>> This is usefull in some network with firewall blocking git port.
>>>>
>>>> When BB_GIT_PROTOCOL_FIREWALL is set, the PROTOCOL_MIRRORS table is
>>>> used to find the new protocol (http or https) and the new host (if
>>>> different) to used.
>>>>
>>>> BB_GIT_PROTOCOL_FIREWALL can also contains a list of host accepted.
>>>> This is usefull for exemple, if we use local git repository inside
>>>> the network.
>>>>
>>>> Exemple usage:
>>>>     # Avoid all git protocol
>>>>     BB_GIT_PROTOCOL_FIREWALL = "1"
>>>>
>>>>     # Avoid git protocol, except for srv1 and srv2 in local network
>>>>     BB_GIT_PROTOCOL_FIREWALL = "srv1.mydomain.com;srv2.mydomain.com"
>>>
>>> Why not use the more generic MIRRORS and PREMIRRORS for this? If they
>>> can't support this, we likely should figure out a way to make them work
>>> for it rather than adding new fetcher specific variables.
>>
>> Sorry for resurrecting such an old thread, but I'm wondering if
>> there's been any more work on an upstreamable solution?
>>
>> The problem being that recipes which define SRC_URI to git://* with
>> protocol=git (or with protocol unspecified, since git is the default)
>> fail during fetching for users working behind a firewall which blocks
>> the native git protocol.
>>
>> Although MIRRORS and PREMIRRORS allow mapping from one host to another
>> there's no support for mapping one git fetch protocol to another and
>> it looks like extending the MIRRORS syntax to support that would be a
>> non-trivial change...
>
> Won't the following work:
>
> PREMIRRORS_append = " \
>      git://.*/.* git://HOST/PATH;protocol=https \n \
>      git://.*/.* git://HOST/PATH;protocol=http \n \
> "

Thanks! Yes, that does work. It's the first time I've ever seen HOST
or PATH being used as mirror replacement patterns. Useful to know.

Would a patch adding something similar to the default MIRRORS (ie try
http[s] as a fallback if the git protocol fails) be acceptable in
upstream oe-core?

> Note: I've not actually tried the above, but I expect it will work.  We do
> something similar to provide local (on-disk) git caches using:
>
> git://.*/.* git://${LAYERDIR}/git/MIRRORNAME;protocol=file
>
> --Mark
>
>>> Cheers,
>>>
>>> Richard
>>> --
>>> _______________________________________________
>>> bitbake-devel mailing list
>>> bitbake-devel at lists.openembedded.org
>>> http://lists.openembedded.org/mailman/listinfo/bitbake-devel
>