[OE-core] [RFC][PATCH 0/6] NPM refactoring

[Stefan Herbrechtsmeier]

Am 25.10.19 um 13:08 schrieb Adrian Bunk:
> On Fri, Oct 25, 2019 at 10:35:20AM +0200, Stefan Herbrechtsmeier wrote:
>> Am 24.10.19 um 17:59 schrieb Richard Purdie:
>>> On Thu, 2019-10-24 at 18:37 +0300, Adrian Bunk wrote:
>>>> On Thu, Oct 24, 2019 at 02:12:43PM +0200, Alexander Kanavin wrote:
>>>>> On Thu, 24 Oct 2019 at 14:02, Stefan Herbrechtsmeier <
>>>>> stefan at herbrechtsmeier.net> wrote:
>>>>>
>>>>>> @Richard: What is your opinion about the per recipe dependency?
>>>>>> Typically OE use one recipe per project. The NPM based solution
>>>>>> handle a
>>>>>> project and all dependencies via one recipe.
>>>>>
>>>>> I don't think it's at all realistic to stick to the 'one recipe per
>>>>> component' in node.js world. A typical 'npm install' can pull down
>>>>> hundreds, or over a thousand dependencies, it's not feasible to
>>>>> have a
>>>>> recipe for each.
>>>>
>>>> Debian has for the perl/python/node/go/rust/haskell ecosystems
>>>> one recipe per component, with ~ 1k recipes each.
>>>
>>> I think we'll have to end up having a smaller number of recipes which
>>> generate multiple packages. That gives a reasonable parsing time at the
>>> expense of having to pre-generate some of the recipe, a bit like the
>>> core perl and python recipes work today. The exact split will depend on
>>> the ecosystems and the "blocks" people tend to build in as its a
>>> compromise between building too much and parsing time.
>>
>> How should this work? Node.js consist of multiple independent packages and
>> every package define its own dependencies. Thereby the dependencies is
>> defined with a version range which often reference a single version.
> 
> Yes, it is a problem that backwards compatibility is frequently broken
> in Node.js packages and just using npm might result in 10 different
> versions of the same package installed and used.

Therefore the distribution could add an compatibility identifier 
(interface version) to the packages and translate the version range of 
the dependencies into a compatibility identifier. The default 
compatibility identifier is the major version of the package. This 
allows the distribution to ship one package per compatibility 
identifier. The distribution could provide multiple compatibility 
identifiers by a single package or introduce additional packages / 
compatibility identifiers based on the major, minor and maybe patch 
version of the package.

This information could and should be shared between different 
distributions to automate the creation of distribution packages from npm 
packages.

> This is a huge mess that either has to be sorted out when packaging
> (as is done in Debian), or you end up with security support basically
> impossible for something very exposed on the internet.

Ack

Regards
   Stefan