[OE-core] [PATCH] cve-update-db: Use std library instead of urllib3

[ChenQi]

I just tired to open the web but failed. The web seems down.

Regards,
Chen Qi

On 07/02/2019 03:39 PM, ChenQi wrote:
> A recent do_populate_cve_db failure:
> https://autobuilder.yoctoproject.org/typhoon/#/builders/23/builds/1005/steps/7/logs/errors 
>
>
> Could you please help look at what's going on?
>
> Regards,
> Chen Qi
>
> On 06/24/2019 05:44 PM, Pierre Le Magourou wrote:
>> From: Pierre Le Magourou <pierre.lemagourou at softbankrobotics.com>
>>
>> urllib3 was used in this recipe but it was not set as a
>> dependency. As it is not specifically needed, rewrite the recipe with
>> urllib from the standard library.
>>
>> Signed-off-by: Pierre Le Magourou 
>> <pierre.lemagourou at softbankrobotics.com>
>> ---
>>   meta/recipes-core/meta/cve-update-db.bb | 10 ++++------
>>   1 file changed, 4 insertions(+), 6 deletions(-)
>>
>> diff --git a/meta/recipes-core/meta/cve-update-db.bb 
>> b/meta/recipes-core/meta/cve-update-db.bb
>> index 522fd23807..1f48820cc6 100644
>> --- a/meta/recipes-core/meta/cve-update-db.bb
>> +++ b/meta/recipes-core/meta/cve-update-db.bb
>> @@ -19,7 +19,7 @@ python do_populate_cve_db() {
>>       Update NVD database with json data feed
>>       """
>>   -    import sqlite3, urllib3, shutil, gzip, re
>> +    import sqlite3, urllib, shutil, gzip, re
>>       from datetime import date
>>         BASE_URL = "https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-"
>> @@ -33,16 +33,14 @@ python do_populate_cve_db() {
>>         initialize_db(c)
>>   -    http = urllib3.PoolManager()
>> -
>>       for year in range(YEAR_START, date.today().year + 1):
>>           year_url = BASE_URL + str(year)
>>           meta_url = year_url + ".meta"
>>           json_url = year_url + ".json.gz"
>>             # Retrieve meta last modified date
>> -        with http.request('GET', meta_url, preload_content=False) as r:
>> -            date_line = str(r.data.splitlines()[0])
>> +        with urllib.request.urlopen(meta_url) as r:
>> +            date_line = str(r.read().splitlines()[0])
>>               last_modified = re.search('lastModifiedDate:(.*)', 
>> date_line).group(1)
>>             # Compare with current db last modified date
>> @@ -50,7 +48,7 @@ python do_populate_cve_db() {
>>           meta = c.fetchone()
>>           if not meta or meta[0] != last_modified:
>>               # Update db with current year json file
>> -            with http.request('GET', json_url, 
>> preload_content=False) as r, open(JSON_TMPFILE, 'wb') as tmpfile:
>> +            with urllib.request.urlopen(json_url) as r, 
>> open(JSON_TMPFILE, 'wb') as tmpfile:
>>                   shutil.copyfileobj(r, tmpfile)
>>               with gzip.open(JSON_TMPFILE, 'rt') as jsonfile:
>>                   update_db(c, jsonfile)
>
>