[bitbake-devel] [master][PATCH] fetch2/wget: support releases from private github repositories

[Ross Burton]

On 14/11/2019 14:21, André Draszik wrote:
> The wget / http fetcher currently doesn't support fetching
> assets attached to releases on private GitHub repositories,
> i.e. release artefacts like
>      https://github.com/<user>/<project>/releases/download/v1.0.0/asset1.txt
> 
> Those are special, in that HTTP basic auth is not used / possible
> on the URL as seen in the GitHub UI, but instead the GitHub API
> must be used for downloading (which does support HTTP basic auth)
> where the URL will be different.
> 
> To be able to access the GitHub API, opportunistic authentication
> (auth-no-challenge) needs to be enabled. Then the API needs to
> be queried for the real URL of the file to be downloaded, and
> finally application/octet-stream must be specified explicitly.
> 
> Note that there is a slight difference in the location of the
> REST API endpoints between GitHub.com and GitHub Enterprise.
> 
>      https://developer.github.com/v3/repos/releases/
>      https://developer.github.com/enterprise/2.19/v3/enterprise-admin/
> 
> As it's impossible to determine if a repository is on GitHub
> or not (considering GitHub Enterprise), and even more so if a
> repository is private or not, a new flag is introduced that
> should be set to "1" - "github_private_asset", e.g.

That's a lot of "special" and "unfortunately" and "but", adding a 
site-specific option and a non-trivial site-specific chunk of code to 
the otherwise generic HTTP fetcher.

I'd suggest implementing this as a new fetcher instead of shoe-horning 
it into the wget fetcher.  Then you can do something like githubasset:// 
in SRC_URI and the fetcher can do the right thing instead of jumping 
through hoops and guessing.

Ross