[bitbake-devel] [PATCH v6 00/12] NPM refactoring

[Jean-Marie LEMETAYER]

Hi folks and happy new year,

These patches are part of a set which are mainly in OE-core.
More infos can be found on the openembedded-core list.

For readability here is a link if you want the history of this patchset:
http://lists.openembedded.org/pipermail/openembedded-core/2019-December/290298.html

--- V4

These patches can be found here:
https://github.com/savoirfairelinux/bitbake/tree/npm-refactoring-v4

 - As suggested by Richard, a new npmsw fetcher have been implemented to handle
   the npm dependencies. Now the fetch of the main package is separated from the
   fetch of its dependencies.

 - Add the support of "proxy" fetchers. These fetchers (npm and npmsw) just
   resolves an url and forwards it to a "raw" fetcher (wget, git, ...).
   For example an npm url like:
     npm://registry...;package=...;version=...
   is converted as:
     https://registry.../...tgz;downloadfilename=npm2/...tgz;sha256sum=...

 - Add more hash functions for checksum verification to be able to support the
   subresource integrity [1] returned by the npm registry. The supported hash
   functions are now: md5, sha1, sha256, sha384, sah512

 - More tests have been added for the npm and the npmsw fetchers including tests
   for the proxy forwarding thing (checksum, mirrors).

1: https://www.w3.org/TR/SRI/

--- V5

These patches can be found here:
https://github.com/savoirfairelinux/bitbake/tree/npm-refactoring-v5

 - As suggested by Christopher, the fetchers now use the shlex.quote() function
   to handle arguments to shell function.

--- V6

These patches can be found here:
https://github.com/savoirfairelinux/bitbake/tree/npm-refactoring-v6

 - Fix an issue in the "refactor checsum verification" patch:
   I forgot to fire the MissingChecksumEvent as it was done before.
   This should fix the "md5sum" errors seen on the CI.

 - As suggested by Peter, I remove the "bitbake: " prefix on the patches.

Jean-Marie LEMETAYER (12):
  utils: add sha384_file and sha512_file functions
  utils: add is_semver function
  fetch2: refactor checksum verification
  fetch2: add more hash functions for checksum verification
  fetch2: allow fetchers to forward the donestamp management
  fetch2: allow fetchers to forward the mirrors management
  fetch2: allow fetchers to forward the done condition
  fetch2/wget: fix downloadfilename parameter
  fetch2/npm: refactor the npm fetcher
  tests/fetch: add npm tests
  fetch2: add the npmsw fetcher
  tests/fetch: add npmsw tests

 lib/bb/fetch2/__init__.py | 218 +++++++++------
 lib/bb/fetch2/npm.py      | 538 +++++++++++++++++++-------------------
 lib/bb/fetch2/npmsw.py    | 255 ++++++++++++++++++
 lib/bb/fetch2/wget.py     |   7 +-
 lib/bb/tests/fetch.py     | 434 ++++++++++++++++++++++++++++++
 lib/bb/utils.py           |  40 +++
 6 files changed, 1131 insertions(+), 361 deletions(-)
 create mode 100644 lib/bb/fetch2/npmsw.py

--
2.20.1