[bitbake-devel] [PATCH] fetch2/npm: fix parsing multiple matching packages

[Jean-marie Lemetayer]

Hi Zoltán,

It seems that this issue have already been fixed by this patch:
https://git.openembedded.org/bitbake/commit/?id=3d319c79981811d3cfd4732885057db4fd5afcc2

Not the same implementation as yours, but it fixes the same issue.

Best regards,
Jean-Marie Lemetayer

----- Mail original -----
De: "bitbake-devel" <bitbake-devel at lists.openembedded.org>
À: "mpvader" <mvader at victronenergy.com>, "bitbake-devel" <bitbake-devel at lists.openembedded.org>
Envoyé: Vendredi 4 Octobre 2019 07:58:56
Objet: Re: [bitbake-devel] [PATCH] fetch2/npm: fix parsing multiple matching packages

Hi Bitbake devs!

The exception with the below "devtool add ..." command still occurs with
bvitbake 1.42.0 used in Yocto 2.7 and is fixed with the patch.

Please consider applying/backporting it.

Best regards,
Zoltán Böszörményi

2018. 02. 19. 9:55 keltezéssel, Böszörményi Zoltán írta:
> 2018-02-11 23:31 keltezéssel, mpvader írta:
>> Replace the self-implemented parser with loading the complete json and then
>> taking the last element from it.
>>
>> This fixes an error fetching the lex-parser package, as a dependency of, for
>> example, jison. Reproduce the error with this command:
>>
>> devtool add "npm://registry.npmjs.org;name=jison;version=0.4.18"
>>
>> _parse_view() didn't properly parse the npm view result [1], resulting in
>> an Exception in the subsequent json.loads() function:
>> Exception: ValueError: Extra data: line 57 column 4 - line 296 column 2 (char 5874 - 29698)
> 
> I can confirm this problem exists in Bitbake 1.36 and goes away with the patch.
> 
> Also tested the patch for the handful of node module recipes
> I have and they still build properly.
> 
>>
>> [1] npm view lex-parser@'0.1.x' --json --registry http://registry.npmjs.org
>>
>> Signed-off-by: mpvader <mvader at victronenergy.com>
> 
> Tested-by: Zoltán Böszörményi <zboszor at pr.hu>
> 
>> ---
>>   lib/bb/fetch2/npm.py | 29 ++++++++---------------------
>>   1 file changed, 8 insertions(+), 21 deletions(-)
>>
>> diff --git a/lib/bb/fetch2/npm.py b/lib/bb/fetch2/npm.py
>> index b5f148c..e257355 100644
>> --- a/lib/bb/fetch2/npm.py
>> +++ b/lib/bb/fetch2/npm.py
>> @@ -146,25 +146,6 @@ class Npm(FetchMethod):
>>           self._unpackdep(ud, ud.pkgname, workobj, unpackdir, dldir, d)
>> -    def _parse_view(self, output):
>> -        '''
>> -        Parse the output of npm view --json; the last JSON result
>> -        is assumed to be the one that we're interested in.
>> -        '''
>> -        pdata = None
>> -        outdeps = {}
>> -        datalines = []
>> -        bracelevel = 0
>> -        for line in output.splitlines():
>> -            if bracelevel:
>> -                datalines.append(line)
>> -            elif '{' in line:
>> -                datalines = []
>> -                datalines.append(line)
>> -            bracelevel = bracelevel + line.count('{') - line.count('}')
>> -        if datalines:
>> -            pdata = json.loads('\n'.join(datalines))
>> -        return pdata
>>       def _getdependencies(self, pkg, data, version, d, ud, optional=False, 
>> fetchedlist=None):
>>           if fetchedlist is None:
>> @@ -175,9 +156,15 @@ class Npm(FetchMethod):
>>           logger.debug(2, "Calling getdeps on %s" % pkg)
>>           fetchcmd = "npm view %s --json --registry %s" % (pkgfullname, ud.registry)
>>           output = runfetchcmd(fetchcmd, d, True)
>> -        pdata = self._parse_view(output)
>> -        if not pdata:
>> +        if not output:
>>               raise FetchError("The command '%s' returned no output" % fetchcmd)
>> +        pdata = json.loads(output)
>> +
>> +        # npm registry will return multiple dicts in a list when there is more than one 
>> matching
>> +        # version. Take the last one.
>> +        if isinstance(pdata, list):
>> +            pdata = pdata[-1]
>> +
>>           if optional:
>>               pkg_os = pdata.get('os', None)
>>               if pkg_os:
>>
> 

-- 
_______________________________________________
bitbake-devel mailing list
bitbake-devel at lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/bitbake-devel