<div dir="ltr"><div>Hi Ross/Richard,</div><div><br></div><div>I'd like this applied to Sumo also. Should I create a new patch and send it to the list, or is there a process for requesting this is cherry-picked across?</div><div><br></div><div>Thanks,</div><div>Ryan.</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, 25 Sep 2019 at 13:24, Ross Burton <<a href="mailto:ross.burton@intel.com">ross.burton@intel.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">As detailed at [1] the XML feeds provided by NIST are being discontinued on<br>
October 9th 2019. As cve-check-tool uses these feeds, cve-check.bbclass will be<br>
inoperable after this date.<br>
<br>
To ensure that cve-check continues working, backport the following commits from<br>
master to move away from the unmaintained cve-check-tool to our own Python code<br>
that fetches the JSON:<br>
<br>
546d14135c5 cve-update-db: New recipe to update CVE database<br>
bc144b028f6 cve-check: Remove dependency to cve-check-tool-native<br>
7f62a20b32a cve-check: Manage CVE_PRODUCT with more than one name<br>
3bf63bc6084 cve-check: Consider CVE that affects versions with less than operator<br>
c0eabd30d7b cve-update-db: Use std library instead of urllib3<br>
27eb839ee65 cve-check: be idiomatic<br>
09be21f4d17 cve-update-db: Manage proxy if needed.<br>
975793e3825 cve-update-db: do_populate_cve_db depends on do_fetch<br>
0325dd72714 cve-update-db: Catch request.urlopen errors.<br>
4078da92b49 cve-check: Depends on cve-update-db-native<br>
f7676e9a38d cve-update-db: Use NVD CPE data to populate PRODUCTS table<br>
bc0195be1b1 cve-check: Update unpatched CVE matching<br>
c807c2a6409 cve-update-db-native: Skip recipe when cve-check class is not loaded.<br>
07bb8b25e17 cve-check: remove redundant readline CVE whitelisting<br>
5388ed6d137 cve-check-tool: remove<br>
270ac00cb43 cve-check.bbclass: initialize to_append<br>
e6bf9000987 cve-check: allow comparison of Vendor as well as Product<br>
91770338f76 cve-update-db-native: use SQL placeholders instead of format strings<br>
7069302a4cc cve-check: Replace CVE_CHECK_CVE_WHITELIST by CVE_CHECK_WHITELIST<br>
78de2cb39d7 cve-update-db-native: Remove hash column from database.<br>
4b301030cf9 cve-update-db-native: use os.path.join instead of +<br>
f0d822fad2a cve-update-db: actually inherit native<br>
b309840b6aa cve-update-db-native: use executemany() to optimise CPE insertion<br>
bb4e53af33d cve-update-db-native: improve metadata parsing<br>
94227459792 cve-update-db-native: clean up JSON fetching<br>
95438d52b73 cve-update-db-native: fix https proxy issues<br>
1f9a963b9ff glibc: exclude child recipes from CVE scanning<br>
<br>
[1] <a href="https://nvd.nist.gov/General/News/XML-Vulnerability-Feed-Retirement" rel="noreferrer" target="_blank">https://nvd.nist.gov/General/News/XML-Vulnerability-Feed-Retirement</a><br>
<br>
Signed-off-by: Ross Burton <<a href="mailto:ross.burton@intel.com" target="_blank">ross.burton@intel.com</a>><br>
---<br>
meta/classes/cve-check.bbclass | 142 +++++++-----<br>
meta/conf/distro/include/maintainers.inc | 1 +<br>
meta/recipes-core/glibc/glibc-locale.inc | 3 +<br>
meta/recipes-core/glibc/glibc-mtrace.inc | 3 +<br>
meta/recipes-core/glibc/glibc-scripts.inc | 3 +<br>
.../recipes-core/meta/<a href="http://cve-update-db-native.bb" rel="noreferrer" target="_blank">cve-update-db-native.bb</a> | 195 ++++++++++++++++<br>
.../cve-check-tool/<a href="http://cve-check-tool_5.6.4.bb" rel="noreferrer" target="_blank">cve-check-tool_5.6.4.bb</a> | 62 -----<br>
...x-freeing-memory-allocated-by-sqlite.patch | 50 ----<br>
...erriding-default-CA-certificate-file.patch | 215 ------------------<br>
...s-in-percent-when-downloading-CVE-db.patch | 135 -----------<br>
...omputed-vs-expected-sha256-digit-str.patch | 52 -----<br>
...heck-for-malloc_trim-before-using-it.patch | 51 -----<br>
12 files changed, 292 insertions(+), 620 deletions(-)<br>
create mode 100644 meta/recipes-core/meta/<a href="http://cve-update-db-native.bb" rel="noreferrer" target="_blank">cve-update-db-native.bb</a><br>
delete mode 100644 meta/recipes-devtools/cve-check-tool/<a href="http://cve-check-tool_5.6.4.bb" rel="noreferrer" target="_blank">cve-check-tool_5.6.4.bb</a><br>
delete mode 100644 meta/recipes-devtools/cve-check-tool/files/0001-Fix-freeing-memory-allocated-by-sqlite.patch<br>
delete mode 100644 meta/recipes-devtools/cve-check-tool/files/0001-curl-allow-overriding-default-CA-certificate-file.patch<br>
delete mode 100644 meta/recipes-devtools/cve-check-tool/files/0001-print-progress-in-percent-when-downloading-CVE-db.patch<br>
delete mode 100644 meta/recipes-devtools/cve-check-tool/files/0001-update-Compare-computed-vs-expected-sha256-digit-str.patch<br>
delete mode 100644 meta/recipes-devtools/cve-check-tool/files/check-for-malloc_trim-before-using-it.patch<br>
<br>
diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass<br>
index 743bc08a4f9..c00d2910be1 100644<br>
--- a/meta/classes/cve-check.bbclass<br>
+++ b/meta/classes/cve-check.bbclass<br>
@@ -26,7 +26,7 @@ CVE_PRODUCT ??= "${BPN}"<br>
CVE_VERSION ??= "${PV}"<br>
<br>
CVE_CHECK_DB_DIR ?= "${DL_DIR}/CVE_CHECK"<br>
-CVE_CHECK_DB_FILE ?= "${CVE_CHECK_DB_DIR}/nvd.db"<br>
+CVE_CHECK_DB_FILE ?= "${CVE_CHECK_DB_DIR}/nvdcve_1.0.db"<br>
<br>
CVE_CHECK_LOG ?= "${T}/cve.log"<br>
CVE_CHECK_TMP_FILE ?= "${TMPDIR}/cve_check"<br>
@@ -37,32 +37,33 @@ CVE_CHECK_COPY_FILES ??= "1"<br>
CVE_CHECK_CREATE_MANIFEST ??= "1"<br>
<br>
# Whitelist for packages (PN)<br>
-CVE_CHECK_PN_WHITELIST = "\<br>
- glibc-locale \<br>
-"<br>
+CVE_CHECK_PN_WHITELIST ?= ""<br>
<br>
-# Whitelist for CVE and version of package<br>
-CVE_CHECK_CVE_WHITELIST = "{\<br>
- 'CVE-2014-2524': ('6.3','5.2',), \<br>
-}"<br>
+# Whitelist for CVE. If a CVE is found, then it is considered patched.<br>
+# The value is a string containing space separated CVE values:<br>
+# <br>
+# CVE_CHECK_WHITELIST = 'CVE-2014-2524 CVE-2018-1234'<br>
+# <br>
+CVE_CHECK_WHITELIST ?= ""<br>
<br>
python do_cve_check () {<br>
"""<br>
Check recipe for patched and unpatched CVEs<br>
"""<br>
<br>
- if os.path.exists(d.getVar("CVE_CHECK_TMP_FILE")):<br>
+ if os.path.exists(d.getVar("CVE_CHECK_DB_FILE")):<br>
patched_cves = get_patches_cves(d)<br>
patched, unpatched = check_cves(d, patched_cves)<br>
if patched or unpatched:<br>
cve_data = get_cve_info(d, patched + unpatched)<br>
cve_write_data(d, patched, unpatched, cve_data)<br>
else:<br>
- bb.note("Failed to update CVE database, skipping CVE check")<br>
+ bb.note("No CVE database found, skipping CVE check")<br>
+<br>
}<br>
<br>
addtask cve_check after do_unpack before do_build<br>
-do_cve_check[depends] = "cve-check-tool-native:do_populate_sysroot cve-check-tool-native:do_populate_cve_db"<br>
+do_cve_check[depends] = "cve-update-db-native:do_populate_cve_db"<br>
do_cve_check[nostamp] = "1"<br>
<br>
python cve_check_cleanup () {<br>
@@ -163,65 +164,94 @@ def get_patches_cves(d):<br>
<br>
def check_cves(d, patched_cves):<br>
"""<br>
- Run cve-check-tool looking for patched and unpatched CVEs.<br>
+ Connect to the NVD database and find unpatched cves.<br>
"""<br>
-<br>
import ast, csv, tempfile, subprocess, io<br>
+ from distutils.version import LooseVersion<br>
<br>
- cves_patched = []<br>
cves_unpatched = []<br>
- bpn = d.getVar("CVE_PRODUCT")<br>
+ # CVE_PRODUCT can contain more than one product (eg. curl/libcurl)<br>
+ products = d.getVar("CVE_PRODUCT").split()<br>
# If this has been unset then we're not scanning for CVEs here (for example, image recipes)<br>
- if not bpn:<br>
+ if not products:<br>
return ([], [])<br>
pv = d.getVar("CVE_VERSION").split("+git")[0]<br>
- cves = " ".join(patched_cves)<br>
- cve_db_dir = d.getVar("CVE_CHECK_DB_DIR")<br>
- cve_whitelist = ast.literal_eval(d.getVar("CVE_CHECK_CVE_WHITELIST"))<br>
- cve_cmd = "cve-check-tool"<br>
- cmd = [cve_cmd, "--no-html", "--skip-update", "--csv", "--not-affected", "-t", "faux", "-d", cve_db_dir]<br>
<br>
# If the recipe has been whitlisted we return empty lists<br>
if d.getVar("PN") in d.getVar("CVE_CHECK_PN_WHITELIST").split():<br>
bb.note("Recipe has been whitelisted, skipping check")<br>
return ([], [])<br>
<br>
- try:<br>
- # Write the faux CSV file to be used with cve-check-tool<br>
- fd, faux = tempfile.mkstemp(prefix="cve-faux-")<br>
- with os.fdopen(fd, "w") as f:<br>
- for pn in bpn.split():<br>
- f.write("%s,%s,%s,\n" % (pn, pv, cves))<br>
- cmd.append(faux)<br>
-<br>
- output = subprocess.check_output(cmd).decode("utf-8")<br>
- bb.debug(2, "Output of command %s:\n%s" % ("\n".join(cmd), output))<br>
- except subprocess.CalledProcessError as e:<br>
- bb.warn("Couldn't check for CVEs: %s (output %s)" % (e, e.output))<br>
- finally:<br>
- os.remove(faux)<br>
-<br>
- for row in csv.reader(io.StringIO(output)):<br>
- # Third row has the unpatched CVEs<br>
- if row[2]:<br>
- for cve in row[2].split():<br>
- # Skip if the CVE has been whitlisted for the current version<br>
- if pv in cve_whitelist.get(cve,[]):<br>
- bb.note("%s-%s has been whitelisted for %s" % (bpn, pv, cve))<br>
+ old_cve_whitelist = d.getVar("CVE_CHECK_CVE_WHITELIST")<br>
+ if old_cve_whitelist:<br>
+ bb.warn("CVE_CHECK_CVE_WHITELIST is deprecated, please use CVE_CHECK_WHITELIST.")<br>
+ cve_whitelist = d.getVar("CVE_CHECK_WHITELIST").split()<br>
+<br>
+ import sqlite3<br>
+ db_file = d.getVar("CVE_CHECK_DB_FILE")<br>
+ conn = sqlite3.connect(db_file)<br>
+<br>
+ for product in products:<br>
+ c = conn.cursor()<br>
+ if ":" in product:<br>
+ vendor, product = product.split(":", 1)<br>
+ c.execute("SELECT * FROM PRODUCTS WHERE PRODUCT IS ? AND VENDOR IS ?", (product, vendor))<br>
+ else:<br>
+ c.execute("SELECT * FROM PRODUCTS WHERE PRODUCT IS ?", (product,))<br>
+<br>
+ for row in c:<br>
+ cve = row[0]<br>
+ version_start = row[3]<br>
+ operator_start = row[4]<br>
+ version_end = row[5]<br>
+ operator_end = row[6]<br>
+<br>
+ if cve in cve_whitelist:<br>
+ bb.note("%s-%s has been whitelisted for %s" % (product, pv, cve))<br>
+ elif cve in patched_cves:<br>
+ bb.note("%s has been patched" % (cve))<br>
+ else:<br>
+ to_append = False<br>
+ if (operator_start == '=' and pv == version_start):<br>
+ cves_unpatched.append(cve)<br>
else:<br>
+ if operator_start:<br>
+ try:<br>
+ to_append_start = (operator_start == '>=' and LooseVersion(pv) >= LooseVersion(version_start))<br>
+ to_append_start |= (operator_start == '>' and LooseVersion(pv) > LooseVersion(version_start))<br>
+ except:<br>
+ bb.note("%s: Failed to compare %s %s %s for %s" %<br>
+ (product, pv, operator_start, version_start, cve))<br>
+ to_append_start = False<br>
+ else:<br>
+ to_append_start = False<br>
+<br>
+ if operator_end:<br>
+ try:<br>
+ to_append_end = (operator_end == '<=' and LooseVersion(pv) <= LooseVersion(version_end))<br>
+ to_append_end |= (operator_end == '<' and LooseVersion(pv) < LooseVersion(version_end))<br>
+ except:<br>
+ bb.note("%s: Failed to compare %s %s %s for %s" %<br>
+ (product, pv, operator_end, version_end, cve))<br>
+ to_append_end = False<br>
+ else:<br>
+ to_append_end = False<br>
+<br>
+ if operator_start and operator_end:<br>
+ to_append = to_append_start and to_append_end<br>
+ else:<br>
+ to_append = to_append_start or to_append_end<br>
+<br>
+ if to_append:<br>
cves_unpatched.append(cve)<br>
- bb.debug(2, "%s-%s is not patched for %s" % (bpn, pv, cve))<br>
- # Fourth row has patched CVEs<br>
- if row[3]:<br>
- for cve in row[3].split():<br>
- cves_patched.append(cve)<br>
- bb.debug(2, "%s-%s is patched for %s" % (bpn, pv, cve))<br>
+ bb.debug(2, "%s-%s is not patched for %s" % (product, pv, cve))<br>
+ conn.close()<br>
<br>
- return (cves_patched, cves_unpatched)<br>
+ return (list(patched_cves), cves_unpatched)<br>
<br>
def get_cve_info(d, cves):<br>
"""<br>
- Get CVE information from the database used by cve-check-tool.<br>
+ Get CVE information from the database.<br>
<br>
Unfortunately the only way to get CVE info is set the output to<br>
html (hard to parse) or query directly the database.<br>
@@ -241,9 +271,10 @@ def get_cve_info(d, cves):<br>
for row in cur.execute(query, tuple(cves)):<br>
cve_data[row[0]] = {}<br>
cve_data[row[0]]["summary"] = row[1]<br>
- cve_data[row[0]]["score"] = row[2]<br>
- cve_data[row[0]]["modified"] = row[3]<br>
- cve_data[row[0]]["vector"] = row[4]<br>
+ cve_data[row[0]]["scorev2"] = row[2]<br>
+ cve_data[row[0]]["scorev3"] = row[3]<br>
+ cve_data[row[0]]["modified"] = row[4]<br>
+ cve_data[row[0]]["vector"] = row[5]<br>
conn.close()<br>
<br>
return cve_data<br>
@@ -270,7 +301,8 @@ def cve_write_data(d, patched, unpatched, cve_data):<br>
unpatched_cves.append(cve)<br>
write_string += "CVE STATUS: Unpatched\n"<br>
write_string += "CVE SUMMARY: %s\n" % cve_data[cve]["summary"]<br>
- write_string += "CVSS v2 BASE SCORE: %s\n" % cve_data[cve]["score"]<br>
+ write_string += "CVSS v2 BASE SCORE: %s\n" % cve_data[cve]["scorev2"]<br>
+ write_string += "CVSS v3 BASE SCORE: %s\n" % cve_data[cve]["scorev3"]<br>
write_string += "VECTOR: %s\n" % cve_data[cve]["vector"]<br>
write_string += "MORE INFORMATION: %s%s\n\n" % (nvd_link, cve)<br>
<br>
diff --git a/meta/conf/distro/include/maintainers.inc b/meta/conf/distro/include/maintainers.inc<br>
index 672f0677922..c027901fdf0 100644<br>
--- a/meta/conf/distro/include/maintainers.inc<br>
+++ b/meta/conf/distro/include/maintainers.inc<br>
@@ -116,6 +116,7 @@ RECIPE_MAINTAINER_pn-cryptodev-tests = "Robert Yang <<a href="mailto:liezhi.yang@windriver.com" target="_blank">liezhi.yang@windriver.com</a>>"<br>
RECIPE_MAINTAINER_pn-cups = "Chen Qi <<a href="mailto:Qi.Chen@windriver.com" target="_blank">Qi.Chen@windriver.com</a>>"<br>
RECIPE_MAINTAINER_pn-curl = "Armin Kuster <<a href="mailto:akuster808@gmail.com" target="_blank">akuster808@gmail.com</a>>"<br>
RECIPE_MAINTAINER_pn-cve-check-tool = "Ross Burton <<a href="mailto:ross.burton@intel.com" target="_blank">ross.burton@intel.com</a>>"<br>
+RECIPE_MAINTAINER_pn-cve-update-db-native = "Ross Burton <<a href="mailto:ross.burton@intel.com" target="_blank">ross.burton@intel.com</a>>"<br>
RECIPE_MAINTAINER_pn-cwautomacros = "Ross Burton <<a href="mailto:ross.burton@intel.com" target="_blank">ross.burton@intel.com</a>>"<br>
RECIPE_MAINTAINER_pn-db = "Mark Hatle <<a href="mailto:mark.hatle@windriver.com" target="_blank">mark.hatle@windriver.com</a>>"<br>
RECIPE_MAINTAINER_pn-dbus = "Chen Qi <<a href="mailto:Qi.Chen@windriver.com" target="_blank">Qi.Chen@windriver.com</a>>"<br>
diff --git a/meta/recipes-core/glibc/glibc-locale.inc b/meta/recipes-core/glibc/glibc-locale.inc<br>
index 1b676dc26e7..97d83cb856d 100644<br>
--- a/meta/recipes-core/glibc/glibc-locale.inc<br>
+++ b/meta/recipes-core/glibc/glibc-locale.inc<br>
@@ -95,3 +95,6 @@ do_install () {<br>
inherit libc-package<br>
<br>
BBCLASSEXTEND = "nativesdk"<br>
+<br>
+# Don't scan for CVEs as glibc will be scanned<br>
+CVE_PRODUCT = ""<br>
diff --git a/meta/recipes-core/glibc/glibc-mtrace.inc b/meta/recipes-core/glibc/glibc-mtrace.inc<br>
index d703c14bdc1..ef9d60ec239 100644<br>
--- a/meta/recipes-core/glibc/glibc-mtrace.inc<br>
+++ b/meta/recipes-core/glibc/glibc-mtrace.inc<br>
@@ -11,3 +11,6 @@ do_install() {<br>
install -d -m 0755 ${D}${bindir}<br>
install -m 0755 ${SRC}/mtrace ${D}${bindir}/<br>
}<br>
+<br>
+# Don't scan for CVEs as glibc will be scanned<br>
+CVE_PRODUCT = ""<br>
diff --git a/meta/recipes-core/glibc/glibc-scripts.inc b/meta/recipes-core/glibc/glibc-scripts.inc<br>
index 2a2b41507ed..14a14e45126 100644<br>
--- a/meta/recipes-core/glibc/glibc-scripts.inc<br>
+++ b/meta/recipes-core/glibc/glibc-scripts.inc<br>
@@ -18,3 +18,6 @@ do_install() {<br>
# sotruss script requires sotruss-lib.so (given by libsotruss package), <br>
# to produce trace of the library calls.<br>
RDEPENDS_${PN} += "libsotruss"<br>
+<br>
+# Don't scan for CVEs as glibc will be scanned<br>
+CVE_PRODUCT = ""<br>
diff --git a/meta/recipes-core/meta/<a href="http://cve-update-db-native.bb" rel="noreferrer" target="_blank">cve-update-db-native.bb</a> b/meta/recipes-core/meta/<a href="http://cve-update-db-native.bb" rel="noreferrer" target="_blank">cve-update-db-native.bb</a><br>
new file mode 100644<br>
index 00000000000..2c427a5884f<br>
--- /dev/null<br>
+++ b/meta/recipes-core/meta/<a href="http://cve-update-db-native.bb" rel="noreferrer" target="_blank">cve-update-db-native.bb</a><br>
@@ -0,0 +1,195 @@<br>
+SUMMARY = "Updates the NVD CVE database"<br>
+LICENSE = "MIT"<br>
+<br>
+INHIBIT_DEFAULT_DEPS = "1"<br>
+<br>
+inherit native<br>
+<br>
+deltask do_unpack<br>
+deltask do_patch<br>
+deltask do_configure<br>
+deltask do_compile<br>
+deltask do_install<br>
+deltask do_populate_sysroot<br>
+<br>
+python () {<br>
+ if not d.getVar("CVE_CHECK_DB_FILE"):<br>
+ raise bb.parse.SkipRecipe("Skip recipe when cve-check class is not loaded.")<br>
+}<br>
+<br>
+python do_populate_cve_db() {<br>
+ """<br>
+ Update NVD database with json data feed<br>
+ """<br>
+<br>
+ import sqlite3, urllib, urllib.parse, shutil, gzip<br>
+ from datetime import date<br>
+<br>
+ BASE_URL = "<a href="https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-" rel="noreferrer" target="_blank">https://nvd.nist.gov/feeds/json/cve/1.0/nvdcve-1.0-</a>"<br>
+ YEAR_START = 2002<br>
+<br>
+ db_dir = os.path.join(d.getVar("DL_DIR"), 'CVE_CHECK')<br>
+ db_file = os.path.join(db_dir, 'nvdcve_1.0.db')<br>
+ json_tmpfile = os.path.join(db_dir, 'nvd.json.gz')<br>
+ proxy = d.getVar("https_proxy")<br>
+<br>
+ if proxy:<br>
+ # instantiate an opener but do not install it as the global<br>
+ # opener unless if we're really sure it's applicable for all<br>
+ # urllib requests<br>
+ proxy_handler = urllib.request.ProxyHandler({'https': proxy})<br>
+ proxy_opener = urllib.request.build_opener(proxy_handler)<br>
+ else:<br>
+ proxy_opener = None<br>
+<br>
+ cve_f = open(os.path.join(d.getVar("TMPDIR"), 'cve_check'), 'a')<br>
+<br>
+ if not os.path.isdir(db_dir):<br>
+ os.mkdir(db_dir)<br>
+<br>
+ # Connect to database<br>
+ conn = sqlite3.connect(db_file)<br>
+ c = conn.cursor()<br>
+<br>
+ initialize_db(c)<br>
+<br>
+ for year in range(YEAR_START, date.today().year + 1):<br>
+ year_url = BASE_URL + str(year)<br>
+ meta_url = year_url + ".meta"<br>
+ json_url = year_url + ".json.gz"<br>
+<br>
+ # Retrieve meta last modified date<br>
+<br>
+ response = None<br>
+<br>
+ if proxy_opener:<br>
+ response = proxy_opener.open(meta_url)<br>
+ else:<br>
+ req = urllib.request.Request(meta_url)<br>
+ response = urllib.request.urlopen(req)<br>
+<br>
+ if response:<br>
+ for l in response.read().decode("utf-8").splitlines():<br>
+ key, value = l.split(":", 1)<br>
+ if key == "lastModifiedDate":<br>
+ last_modified = value<br>
+ break<br>
+ else:<br>
+ bb.warn("Cannot parse CVE metadata, update failed")<br>
+ return<br>
+<br>
+ # Compare with current db last modified date<br>
+ c.execute("select DATE from META where YEAR = ?", (year,))<br>
+ meta = c.fetchone()<br>
+ if not meta or meta[0] != last_modified:<br>
+ # Clear products table entries corresponding to current year<br>
+ c.execute("delete from PRODUCTS where ID like ?", ('CVE-%d%%' % year,))<br>
+<br>
+ # Update db with current year json file<br>
+ try:<br>
+ if proxy_opener:<br>
+ response = proxy_opener.open(json_url)<br>
+ else:<br>
+ req = urllib.request.Request(json_url)<br>
+ response = urllib.request.urlopen(req)<br>
+<br>
+ if response:<br>
+ update_db(c, gzip.decompress(response.read()).decode('utf-8'))<br>
+ c.execute("insert or replace into META values (?, ?)", [year, last_modified])<br>
+ except urllib.error.URLError as e:<br>
+ cve_f.write('Warning: CVE db update error, CVE data is outdated.\n\n')<br>
+ bb.warn("Cannot parse CVE data (%s), update failed" % e.reason)<br>
+ return<br>
+<br>
+ # Update success, set the date to cve_check file.<br>
+ if year == date.today().year:<br>
+ cve_f.write('CVE database update : %s\n\n' % date.today())<br>
+<br>
+ cve_f.close()<br>
+ conn.commit()<br>
+ conn.close()<br>
+}<br>
+<br>
+def initialize_db(c):<br>
+ c.execute("CREATE TABLE IF NOT EXISTS META (YEAR INTEGER UNIQUE, DATE TEXT)")<br>
+ c.execute("CREATE TABLE IF NOT EXISTS NVD (ID TEXT UNIQUE, SUMMARY TEXT, \<br>
+ SCOREV2 TEXT, SCOREV3 TEXT, MODIFIED INTEGER, VECTOR TEXT)")<br>
+ c.execute("CREATE TABLE IF NOT EXISTS PRODUCTS (ID TEXT, \<br>
+ VENDOR TEXT, PRODUCT TEXT, VERSION_START TEXT, OPERATOR_START TEXT, \<br>
+ VERSION_END TEXT, OPERATOR_END TEXT)")<br>
+<br>
+def parse_node_and_insert(c, node, cveId):<br>
+ # Parse children node if needed<br>
+ for child in node.get('children', ()):<br>
+ parse_node_and_insert(c, child, cveId)<br>
+<br>
+ def cpe_generator():<br>
+ for cpe in node.get('cpe_match', ()):<br>
+ if not cpe['vulnerable']:<br>
+ return<br>
+ cpe23 = cpe['cpe23Uri'].split(':')<br>
+ vendor = cpe23[3]<br>
+ product = cpe23[4]<br>
+ version = cpe23[5]<br>
+<br>
+ if version != '*':<br>
+ # Version is defined, this is a '=' match<br>
+ yield [cveId, vendor, product, version, '=', '', '']<br>
+ else:<br>
+ # Parse start version, end version and operators<br>
+ op_start = ''<br>
+ op_end = ''<br>
+ v_start = ''<br>
+ v_end = ''<br>
+<br>
+ if 'versionStartIncluding' in cpe:<br>
+ op_start = '>='<br>
+ v_start = cpe['versionStartIncluding']<br>
+<br>
+ if 'versionStartExcluding' in cpe:<br>
+ op_start = '>'<br>
+ v_start = cpe['versionStartExcluding']<br>
+<br>
+ if 'versionEndIncluding' in cpe:<br>
+ op_end = '<='<br>
+ v_end = cpe['versionEndIncluding']<br>
+<br>
+ if 'versionEndExcluding' in cpe:<br>
+ op_end = '<'<br>
+ v_end = cpe['versionEndExcluding']<br>
+<br>
+ yield [cveId, vendor, product, v_start, op_start, v_end, op_end]<br>
+<br>
+ c.executemany("insert into PRODUCTS values (?, ?, ?, ?, ?, ?, ?)", cpe_generator())<br>
+<br>
+def update_db(c, jsondata):<br>
+ import json<br>
+ root = json.loads(jsondata)<br>
+<br>
+ for elt in root['CVE_Items']:<br>
+ if not elt['impact']:<br>
+ continue<br>
+<br>
+ cveId = elt['cve']['CVE_data_meta']['ID']<br>
+ cveDesc = elt['cve']['description']['description_data'][0]['value']<br>
+ date = elt['lastModifiedDate']<br>
+ accessVector = elt['impact']['baseMetricV2']['cvssV2']['accessVector']<br>
+ cvssv2 = elt['impact']['baseMetricV2']['cvssV2']['baseScore']<br>
+<br>
+ try:<br>
+ cvssv3 = elt['impact']['baseMetricV3']['cvssV3']['baseScore']<br>
+ except:<br>
+ cvssv3 = 0.0<br>
+<br>
+ c.execute("insert or replace into NVD values (?, ?, ?, ?, ?, ?)",<br>
+ [cveId, cveDesc, cvssv2, cvssv3, date, accessVector])<br>
+<br>
+ configurations = elt['configurations']['nodes']<br>
+ for config in configurations:<br>
+ parse_node_and_insert(c, config, cveId)<br>
+<br>
+<br>
+addtask do_populate_cve_db before do_fetch<br>
+do_populate_cve_db[nostamp] = "1"<br>
+<br>
+EXCLUDE_FROM_WORLD = "1"<br>
diff --git a/meta/recipes-devtools/cve-check-tool/<a href="http://cve-check-tool_5.6.4.bb" rel="noreferrer" target="_blank">cve-check-tool_5.6.4.bb</a> b/meta/recipes-devtools/cve-check-tool/<a href="http://cve-check-tool_5.6.4.bb" rel="noreferrer" target="_blank">cve-check-tool_5.6.4.bb</a><br>
deleted file mode 100644<br>
index 1c84fb1cf2d..00000000000<br>
--- a/meta/recipes-devtools/cve-check-tool/<a href="http://cve-check-tool_5.6.4.bb" rel="noreferrer" target="_blank">cve-check-tool_5.6.4.bb</a><br>
+++ /dev/null<br>
@@ -1,62 +0,0 @@<br>
-SUMMARY = "cve-check-tool"<br>
-DESCRIPTION = "cve-check-tool is a tool for checking known (public) CVEs.\<br>
-The tool will identify potentially vunlnerable software packages within Linux distributions through version matching."<br>
-HOMEPAGE = "<a href="https://github.com/ikeydoherty/cve-check-tool" rel="noreferrer" target="_blank">https://github.com/ikeydoherty/cve-check-tool</a>"<br>
-SECTION = "Development/Tools"<br>
-LICENSE = "GPL-2.0+"<br>
-LIC_FILES_CHKSUM = "file://LICENSE;md5=e8c1458438ead3c34974bc0be3a03ed6"<br>
-<br>
-SRC_URI = "<a href="https://github.com/ikeydoherty/$%7BBPN%7D/releases/download/v$%7BPV%7D/$%7BBP%7D.tar.xz" rel="noreferrer" target="_blank">https://github.com/ikeydoherty/${BPN}/releases/download/v${PV}/${BP}.tar.xz</a> \<br>
- file://check-for-malloc_trim-before-using-it.patch \<br>
- file://0001-print-progress-in-percent-when-downloading-CVE-db.patch \<br>
- file://0001-curl-allow-overriding-default-CA-certificate-file.patch \<br>
- file://0001-update-Compare-computed-vs-expected-sha256-digit-str.patch \<br>
- file://0001-Fix-freeing-memory-allocated-by-sqlite.patch \<br>
- "<br>
-<br>
-SRC_URI[md5sum] = "c5f4247140fc9be3bf41491d31a34155"<br>
-SRC_URI[sha256sum] = "b8f283be718af8d31232ac1bfc10a0378fb958aaaa49af39168f8acf501e6a5b"<br>
-<br>
-UPSTREAM_CHECK_URI = "<a href="https://github.com/ikeydoherty/cve-check-tool/releases" rel="noreferrer" target="_blank">https://github.com/ikeydoherty/cve-check-tool/releases</a>"<br>
-<br>
-DEPENDS = "libcheck glib-2.0 json-glib curl libxml2 sqlite3 openssl ca-certificates"<br>
-<br>
-RDEPENDS_${PN} = "ca-certificates"<br>
-<br>
-inherit pkgconfig autotools<br>
-<br>
-EXTRA_OECONF = "--disable-coverage --enable-relative-plugins"<br>
-CFLAGS_append = " -Wno-error=pedantic"<br>
-<br>
-do_populate_cve_db() {<br>
- if [ "${BB_NO_NETWORK}" = "1" ] ; then<br>
- bbwarn "BB_NO_NETWORK is set; Can't update cve-check-tool database, new CVEs won't be detected"<br>
- return<br>
- fi<br>
-<br>
- # In case we don't inherit cve-check class, use default values defined in the class.<br>
- cve_dir="${CVE_CHECK_DB_DIR}"<br>
- cve_file="${CVE_CHECK_TMP_FILE}"<br>
-<br>
- [ -z "${cve_dir}" ] && cve_dir="${DL_DIR}/CVE_CHECK"<br>
- [ -z "${cve_file}" ] && cve_file="${TMPDIR}/cve_check"<br>
-<br>
- unused="${@bb.utils.export_proxies(d)}"<br>
- bbdebug 2 "Updating cve-check-tool database located in $cve_dir"<br>
- # --cacert works around curl-native not finding the CA bundle<br>
- if cve-check-update --cacert ${sysconfdir}/ssl/certs/ca-certificates.crt -d "$cve_dir" ; then<br>
- printf "CVE database was updated on %s UTC\n\n" "$(LANG=C date --utc +'%F %T')" > "$cve_file"<br>
- else<br>
- bbwarn "Error in executing cve-check-update"<br>
- if [ "${@'1' if bb.data.inherits_class('cve-check', d) else '0'}" -ne 0 ] ; then<br>
- bbwarn "Failed to update cve-check-tool database, CVEs won't be checked"<br>
- fi<br>
- fi<br>
-}<br>
-<br>
-addtask populate_cve_db after do_populate_sysroot<br>
-do_populate_cve_db[depends] = "cve-check-tool-native:do_populate_sysroot"<br>
-do_populate_cve_db[nostamp] = "1"<br>
-do_populate_cve_db[progress] = "percent"<br>
-<br>
-BBCLASSEXTEND = "native nativesdk"<br>
diff --git a/meta/recipes-devtools/cve-check-tool/files/0001-Fix-freeing-memory-allocated-by-sqlite.patch b/meta/recipes-devtools/cve-check-tool/files/0001-Fix-freeing-memory-allocated-by-sqlite.patch<br>
deleted file mode 100644<br>
index 4a82cf2dded..00000000000<br>
--- a/meta/recipes-devtools/cve-check-tool/files/0001-Fix-freeing-memory-allocated-by-sqlite.patch<br>
+++ /dev/null<br>
@@ -1,50 +0,0 @@<br>
-From a3353429652f83bb8b0316500faa88fa2555542d Mon Sep 17 00:00:00 2001<br>
-From: Peter Marko <<a href="mailto:peter.marko@siemens.com" target="_blank">peter.marko@siemens.com</a>><br>
-Date: Thu, 13 Apr 2017 23:09:52 +0200<br>
-Subject: [PATCH] Fix freeing memory allocated by sqlite<br>
-<br>
-Upstream-Status: Backport<br>
-Signed-off-by: Peter Marko <<a href="mailto:peter.marko@siemens.com" target="_blank">peter.marko@siemens.com</a>><br>
----<br>
- src/core.c | 8 ++++----<br>
- 1 file changed, 4 insertions(+), 4 deletions(-)<br>
-<br>
-diff --git a/src/core.c b/src/core.c<br>
-index 6263031..6788f16 100644<br>
---- a/src/core.c<br>
-+++ b/src/core.c<br>
-@@ -82,7 +82,7 @@ static bool ensure_table(CveDB *self)<br>
- rc = sqlite3_exec(self->db, query, NULL, NULL, &err);<br>
- if (rc != SQLITE_OK) {<br>
- fprintf(stderr, "ensure_table(): %s\n", err);<br>
-- free(err);<br>
-+ sqlite3_free(err);<br>
- return false;<br>
- }<br>
- <br>
-@@ -91,7 +91,7 @@ static bool ensure_table(CveDB *self)<br>
- rc = sqlite3_exec(self->db, query, NULL, NULL, &err);<br>
- if (rc != SQLITE_OK) {<br>
- fprintf(stderr, "ensure_table(): %s\n", err);<br>
-- free(err);<br>
-+ sqlite3_free(err);<br>
- return false;<br>
- }<br>
- <br>
-@@ -99,11 +99,11 @@ static bool ensure_table(CveDB *self)<br>
- rc = sqlite3_exec(self->db, query, NULL, NULL, &err);<br>
- if (rc != SQLITE_OK) {<br>
- fprintf(stderr, "ensure_table(): %s\n", err);<br>
-- free(err);<br>
-+ sqlite3_free(err);<br>
- return false;<br>
- }<br>
- if (err) {<br>
-- free(err);<br>
-+ sqlite3_free(err);<br>
- }<br>
- <br>
- return true;<br>
--- <br>
-2.1.4<br>
-<br>
diff --git a/meta/recipes-devtools/cve-check-tool/files/0001-curl-allow-overriding-default-CA-certificate-file.patch b/meta/recipes-devtools/cve-check-tool/files/0001-curl-allow-overriding-default-CA-certificate-file.patch<br>
deleted file mode 100644<br>
index 3d8ebd1bd26..00000000000<br>
--- a/meta/recipes-devtools/cve-check-tool/files/0001-curl-allow-overriding-default-CA-certificate-file.patch<br>
+++ /dev/null<br>
@@ -1,215 +0,0 @@<br>
-From 825a9969dea052b02ba868bdf39e676349f10dce Mon Sep 17 00:00:00 2001<br>
-From: Jussi Kukkonen <<a href="mailto:jussi.kukkonen@intel.com" target="_blank">jussi.kukkonen@intel.com</a>><br>
-Date: Thu, 9 Feb 2017 14:51:28 +0200<br>
-Subject: [PATCH] curl: allow overriding default CA certificate file<br>
-<br>
-Similar to curl, --cacert can now be used in cve-check-tool and<br>
-cve-check-update to override the default CA certificate file. Useful<br>
-in cases where the system default is unsuitable (for example,<br>
-out-dated) or broken (as in OE's current native libcurl, which embeds<br>
-a path string from one build host and then uses it on another although<br>
-the right path may have become something different).<br>
-<br>
-Upstream-Status: Submitted [<a href="https://github.com/ikeydoherty/cve-check-tool/pull/45" rel="noreferrer" target="_blank">https://github.com/ikeydoherty/cve-check-tool/pull/45</a>]<br>
-<br>
-Signed-off-by: Patrick Ohly <<a href="mailto:patrick.ohly@intel.com" target="_blank">patrick.ohly@intel.com</a>><br>
-<br>
-<br>
-Took Patrick Ohlys original patch from meta-security-isafw, rebased<br>
-on top of other patches.<br>
-<br>
-Signed-off-by: Jussi Kukkonen <<a href="mailto:jussi.kukkonen@intel.com" target="_blank">jussi.kukkonen@intel.com</a>><br>
----<br>
- src/library/cve-check-tool.h | 1 +<br>
- src/library/fetch.c | 10 +++++++++-<br>
- src/library/fetch.h | 3 ++-<br>
- src/main.c | 5 ++++-<br>
- src/update-main.c | 4 +++-<br>
- src/update.c | 12 +++++++-----<br>
- src/update.h | 2 +-<br>
- 7 files changed, 27 insertions(+), 10 deletions(-)<br>
-<br>
-diff --git a/src/library/cve-check-tool.h b/src/library/cve-check-tool.h<br>
-index e4bb5b1..f89eade 100644<br>
---- a/src/library/cve-check-tool.h<br>
-+++ b/src/library/cve-check-tool.h<br>
-@@ -43,6 +43,7 @@ typedef struct CveCheckTool {<br>
- bool bugs; /**<Whether bug tracking is enabled */<br>
- GHashTable *mapping; /**<CVE Mapping */<br>
- const char *output_file; /**<Output file, if any */<br>
-+ const char *cacert_file; /**<Non-default SSL certificate file, if any */<br>
- } CveCheckTool;<br>
- <br>
- /**<br>
-diff --git a/src/library/fetch.c b/src/library/fetch.c<br>
-index 0fe6d76..8f998c3 100644<br>
---- a/src/library/fetch.c<br>
-+++ b/src/library/fetch.c<br>
-@@ -60,7 +60,8 @@ static int progress_callback_new(void *ptr, curl_off_t dltotal, curl_off_t dlnow<br>
- }<br>
- <br>
- FetchStatus fetch_uri(const char *uri, const char *target, bool verbose,<br>
-- unsigned int start_percent, unsigned int end_percent)<br>
-+ unsigned int start_percent, unsigned int end_percent,<br>
-+ const char *cacert_file)<br>
- {<br>
- FetchStatus ret = FETCH_STATUS_FAIL;<br>
- CURLcode res;<br>
-@@ -74,6 +75,13 @@ FetchStatus fetch_uri(const char *uri, const char *target, bool verbose,<br>
- return ret;<br>
- }<br>
- <br>
-+ if (cacert_file) {<br>
-+ res = curl_easy_setopt(curl, CURLOPT_CAINFO, cacert_file);<br>
-+ if (res != CURLE_OK) {<br>
-+ goto bail;<br>
-+ }<br>
-+ }<br>
-+<br>
- if (stat(target, &st) == 0) {<br>
- res = curl_easy_setopt(curl, CURLOPT_TIMECONDITION, CURL_TIMECOND_IFMODSINCE);<br>
- if (res != CURLE_OK) {<br>
-diff --git a/src/library/fetch.h b/src/library/fetch.h<br>
-index 4cce5d1..836c7d7 100644<br>
---- a/src/library/fetch.h<br>
-+++ b/src/library/fetch.h<br>
-@@ -29,7 +29,8 @@ typedef enum {<br>
- * @return A FetchStatus, indicating the operation taken<br>
- */<br>
- FetchStatus fetch_uri(const char *uri, const char *target, bool verbose,<br>
-- unsigned int this_percent, unsigned int next_percent);<br>
-+ unsigned int this_percent, unsigned int next_percent,<br>
-+ const char *cacert_file);<br>
- <br>
- /**<br>
- * Attempt to extract the given gzipped file<br>
-diff --git a/src/main.c b/src/main.c<br>
-index 8e6f158..ae69d47 100644<br>
---- a/src/main.c<br>
-+++ b/src/main.c<br>
-@@ -280,6 +280,7 @@ static bool csv_mode = false;<br>
- static char *modified_stamp = NULL;<br>
- static gchar *mapping_file = NULL;<br>
- static gchar *output_file = NULL;<br>
-+static gchar *cacert_file = NULL;<br>
- <br>
- static GOptionEntry _entries[] = {<br>
- { "not-patched", 'n', 0, G_OPTION_ARG_NONE, &hide_patched, "Hide patched/addressed CVEs", NULL },<br>
-@@ -294,6 +295,7 @@ static GOptionEntry _entries[] = {<br>
- { "csv", 'c', 0, G_OPTION_ARG_NONE, &csv_mode, "Output CSV formatted data only", NULL },<br>
- { "mapping", 'M', 0, G_OPTION_ARG_STRING, &mapping_file, "Path to a mapping file", NULL},<br>
- { "output-file", 'o', 0, G_OPTION_ARG_STRING, &output_file, "Path to the output file (output plugin specific)", NULL},<br>
-+ { "cacert", 'C', 0, G_OPTION_ARG_STRING, &cacert_file, "Path to the combined SSL certificates file (system default is used if not set)", NULL},<br>
- { .short_name = 0 }<br>
- };<br>
- <br>
-@@ -492,6 +494,7 @@ int main(int argc, char **argv)<br>
- <br>
- quiet = csv_mode || !no_html;<br>
- self->output_file = output_file;<br>
-+ self->cacert_file = cacert_file;<br>
- <br>
- if (!csv_mode && self->output_file) {<br>
- quiet = false;<br>
-@@ -530,7 +533,7 @@ int main(int argc, char **argv)<br>
- if (status) {<br>
- fprintf(stderr, "Update of db forced\n");<br>
- cve_db_unlock();<br>
-- if (!update_db(quiet, db_path->str)) {<br>
-+ if (!update_db(quiet, db_path->str, self->cacert_file)) {<br>
- fprintf(stderr, "DB update failure\n");<br>
- goto cleanup;<br>
- }<br>
-diff --git a/src/update-main.c b/src/update-main.c<br>
-index 2379cfa..c52d9d0 100644<br>
---- a/src/update-main.c<br>
-+++ b/src/update-main.c<br>
-@@ -43,11 +43,13 @@ the Free Software Foundation; either version 2 of the License, or\n\<br>
- static gchar *nvds = NULL;<br>
- static bool _show_version = false;<br>
- static bool _quiet = false;<br>
-+static const char *_cacert_file = NULL;<br>
- <br>
- static GOptionEntry _entries[] = {<br>
- { "nvd-dir", 'd', 0, G_OPTION_ARG_STRING, &nvds, "NVD directory in filesystem", NULL },<br>
- { "version", 'v', 0, G_OPTION_ARG_NONE, &_show_version, "Show version", NULL },<br>
- { "quiet", 'q', 0, G_OPTION_ARG_NONE, &_quiet, "Run silently", NULL },<br>
-+ { "cacert", 'C', 0, G_OPTION_ARG_STRING, &_cacert_file, "Path to the combined SSL certificates file (system default is used if not set)", NULL},<br>
- { .short_name = 0 }<br>
- };<br>
- <br>
-@@ -88,7 +90,7 @@ int main(int argc, char **argv)<br>
- goto end;<br>
- }<br>
- <br>
-- if (update_db(_quiet, db_path->str)) {<br>
-+ if (update_db(_quiet, db_path->str, _cacert_file)) {<br>
- ret = EXIT_SUCCESS;<br>
- } else {<br>
- fprintf(stderr, "Failed to update database\n");<br>
-diff --git a/src/update.c b/src/update.c<br>
-index 070560a..8cb4a39 100644<br>
---- a/src/update.c<br>
-+++ b/src/update.c<br>
-@@ -267,7 +267,8 @@ static inline void update_end(int fd, const char *update_fname, bool ok)<br>
- <br>
- static int do_fetch_update(int year, const char *db_dir, CveDB *cve_db,<br>
- bool db_exist, bool verbose,<br>
-- unsigned int this_percent, unsigned int next_percent)<br>
-+ unsigned int this_percent, unsigned int next_percent,<br>
-+ const char *cacert_file)<br>
- {<br>
- const char nvd_uri[] = URI_PREFIX;<br>
- autofree(cve_string) *uri_meta = NULL;<br>
-@@ -331,14 +332,14 @@ refetch:<br>
- }<br>
- <br>
- /* Fetch NVD META file */<br>
-- st = fetch_uri(uri_meta->str, nvdcve_meta->str, verbose, this_percent, this_percent);<br>
-+ st = fetch_uri(uri_meta->str, nvdcve_meta->str, verbose, this_percent, this_percent, cacert_file);<br>
- if (st == FETCH_STATUS_FAIL) {<br>
- fprintf(stderr, "Failed to fetch %s\n", uri_meta->str);<br>
- return -1;<br>
- }<br>
- <br>
- /* Fetch NVD XML file */<br>
-- st = fetch_uri(uri_data_gz->str, nvdcve_data_gz->str, verbose, this_percent, next_percent);<br>
-+ st = fetch_uri(uri_data_gz->str, nvdcve_data_gz->str, verbose, this_percent, next_percent, cacert_file);<br>
- switch (st) {<br>
- case FETCH_STATUS_FAIL:<br>
- fprintf(stderr, "Failed to fetch %s\n", uri_data_gz->str);<br>
-@@ -391,7 +392,7 @@ refetch:<br>
- return 0;<br>
- }<br>
- <br>
--bool update_db(bool quiet, const char *db_file)<br>
-+bool update_db(bool quiet, const char *db_file, const char *cacert_file)<br>
- {<br>
- autofree(char) *db_dir = NULL;<br>
- autofree(CveDB) *cve_db = NULL;<br>
-@@ -466,7 +467,8 @@ bool update_db(bool quiet, const char *db_file)<br>
- if (!quiet)<br>
- fprintf(stderr, "completed: %u%%\r", start_percent);<br>
- rc = do_fetch_update(y, db_dir, cve_db, db_exist, !quiet,<br>
-- start_percent, end_percent);<br>
-+ start_percent, end_percent,<br>
-+ cacert_file);<br>
- switch (rc) {<br>
- case 0:<br>
- if (!quiet)<br>
-diff --git a/src/update.h b/src/update.h<br>
-index b8e9911..ceea0c3 100644<br>
---- a/src/update.h<br>
-+++ b/src/update.h<br>
-@@ -15,7 +15,7 @@ cve_string *get_db_path(const char *path);<br>
- <br>
- int update_required(const char *db_file);<br>
- <br>
--bool update_db(bool quiet, const char *db_file);<br>
-+bool update_db(bool quiet, const char *db_file, const char *cacert_file);<br>
- <br>
- <br>
- /*<br>
--- <br>
-2.1.4<br>
-<br>
diff --git a/meta/recipes-devtools/cve-check-tool/files/0001-print-progress-in-percent-when-downloading-CVE-db.patch b/meta/recipes-devtools/cve-check-tool/files/0001-print-progress-in-percent-when-downloading-CVE-db.patch<br>
deleted file mode 100644<br>
index 8ea6f686e3f..00000000000<br>
--- a/meta/recipes-devtools/cve-check-tool/files/0001-print-progress-in-percent-when-downloading-CVE-db.patch<br>
+++ /dev/null<br>
@@ -1,135 +0,0 @@<br>
-From e9ed26cde63f8ca7607a010a518329339f8c02d3 Mon Sep 17 00:00:00 2001<br>
-From: =?UTF-8?q?Andr=C3=A9=20Draszik?= <<a href="mailto:git@andred.net" target="_blank">git@andred.net</a>><br>
-Date: Mon, 26 Sep 2016 12:12:41 +0100<br>
-Subject: [PATCH] print progress in percent when downloading CVE db<br>
-MIME-Version: 1.0<br>
-Content-Type: text/plain; charset=UTF-8<br>
-Content-Transfer-Encoding: 8bit<br>
-<br>
-Upstream-Status: Pending<br>
-Signed-off-by: André Draszik <<a href="mailto:git@andred.net" target="_blank">git@andred.net</a>><br>
----<br>
- src/library/fetch.c | 28 +++++++++++++++++++++++++++-<br>
- src/library/fetch.h | 3 ++-<br>
- src/update.c | 16 ++++++++++++----<br>
- 3 files changed, 41 insertions(+), 6 deletions(-)<br>
-<br>
-diff --git a/src/library/fetch.c b/src/library/fetch.c<br>
-index 06d4b30..0fe6d76 100644<br>
---- a/src/library/fetch.c<br>
-+++ b/src/library/fetch.c<br>
-@@ -37,13 +37,37 @@ static size_t write_func(void *ptr, size_t size, size_t nmemb, struct fetch_t *f<br>
- return fwrite(ptr, size, nmemb, f->f);<br>
- }<br>
- <br>
--FetchStatus fetch_uri(const char *uri, const char *target, bool verbose)<br>
-+struct percent_t {<br>
-+ unsigned int start;<br>
-+ unsigned int end;<br>
-+};<br>
-+<br>
-+static int progress_callback_new(void *ptr, curl_off_t dltotal, curl_off_t dlnow, curl_off_t ultotal, curl_off_t ulnow)<br>
-+{<br>
-+ (void) ultotal;<br>
-+ (void) ulnow;<br>
-+<br>
-+ struct percent_t *percent = (struct percent_t *) ptr;<br>
-+<br>
-+ if (dltotal && percent && percent->end >= percent->start) {<br>
-+ unsigned int diff = percent->end - percent->start;<br>
-+ if (diff) {<br>
-+ fprintf(stderr,"completed: %"CURL_FORMAT_CURL_OFF_T"%%\r", percent->start + (diff * dlnow / dltotal));<br>
-+ }<br>
-+ }<br>
-+<br>
-+ return 0;<br>
-+}<br>
-+<br>
-+FetchStatus fetch_uri(const char *uri, const char *target, bool verbose,<br>
-+ unsigned int start_percent, unsigned int end_percent)<br>
- {<br>
- FetchStatus ret = FETCH_STATUS_FAIL;<br>
- CURLcode res;<br>
- struct stat st;<br>
- CURL *curl = NULL;<br>
- struct fetch_t *f = NULL;<br>
-+ struct percent_t percent = { .start = start_percent, .end = end_percent };<br>
- <br>
- curl = curl_easy_init();<br>
- if (!curl) {<br>
-@@ -67,6 +91,8 @@ FetchStatus fetch_uri(const char *uri, const char *target, bool verbose)<br>
- }<br>
- if (verbose) {<br>
- (void)curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0L);<br>
-+ (void)curl_easy_setopt(curl, CURLOPT_XFERINFODATA, &percent);<br>
-+ (void)curl_easy_setopt(curl, CURLOPT_XFERINFOFUNCTION, progress_callback_new);<br>
- }<br>
- res = curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, (curl_write_callback)write_func);<br>
- if (res != CURLE_OK) {<br>
-diff --git a/src/library/fetch.h b/src/library/fetch.h<br>
-index 70c3779..4cce5d1 100644<br>
---- a/src/library/fetch.h<br>
-+++ b/src/library/fetch.h<br>
-@@ -28,7 +28,8 @@ typedef enum {<br>
- * @param verbose Whether to be verbose<br>
- * @return A FetchStatus, indicating the operation taken<br>
- */<br>
--FetchStatus fetch_uri(const char *uri, const char *target, bool verbose);<br>
-+FetchStatus fetch_uri(const char *uri, const char *target, bool verbose,<br>
-+ unsigned int this_percent, unsigned int next_percent);<br>
- <br>
- /**<br>
- * Attempt to extract the given gzipped file<br>
-diff --git a/src/update.c b/src/update.c<br>
-index 30fbe96..eaeeefd 100644<br>
---- a/src/update.c<br>
-+++ b/src/update.c<br>
-@@ -266,7 +266,8 @@ static inline void update_end(int fd, const char *update_fname, bool ok)<br>
- }<br>
- <br>
- static int do_fetch_update(int year, const char *db_dir, CveDB *cve_db,<br>
-- bool db_exist, bool verbose)<br>
-+ bool db_exist, bool verbose,<br>
-+ unsigned int this_percent, unsigned int next_percent)<br>
- {<br>
- const char nvd_uri[] = URI_PREFIX;<br>
- autofree(cve_string) *uri_meta = NULL;<br>
-@@ -330,14 +331,14 @@ refetch:<br>
- }<br>
- <br>
- /* Fetch NVD META file */<br>
-- st = fetch_uri(uri_meta->str, nvdcve_meta->str, verbose);<br>
-+ st = fetch_uri(uri_meta->str, nvdcve_meta->str, verbose, this_percent, this_percent);<br>
- if (st == FETCH_STATUS_FAIL) {<br>
- fprintf(stderr, "Failed to fetch %s\n", uri_meta->str);<br>
- return -1;<br>
- }<br>
- <br>
- /* Fetch NVD XML file */<br>
-- st = fetch_uri(uri_data_gz->str, nvdcve_data_gz->str, verbose);<br>
-+ st = fetch_uri(uri_data_gz->str, nvdcve_data_gz->str, verbose, this_percent, next_percent);<br>
- switch (st) {<br>
- case FETCH_STATUS_FAIL:<br>
- fprintf(stderr, "Failed to fetch %s\n", uri_data_gz->str);<br>
-@@ -459,10 +460,17 @@ bool update_db(bool quiet, const char *db_file)<br>
- for (int i = YEAR_START; i <= year+1; i++) {<br>
- int y = i > year ? -1 : i;<br>
- int rc;<br>
-+ unsigned int start_percent = ((i+0 - YEAR_START) * 100) / (year+2 - YEAR_START);<br>
-+ unsigned int end_percent = ((i+1 - YEAR_START) * 100) / (year+2 - YEAR_START);<br>
- <br>
-- rc = do_fetch_update(y, db_dir, cve_db, db_exist, !quiet);<br>
-+ if (!quiet)<br>
-+ fprintf(stderr, "completed: %u%%\r", start_percent);<br>
-+ rc = do_fetch_update(y, db_dir, cve_db, db_exist, !quiet,<br>
-+ start_percent, end_percent);<br>
- switch (rc) {<br>
- case 0:<br>
-+ if (!quiet)<br>
-+ fprintf(stderr,"completed: %u%%\r", end_percent);<br>
- continue;<br>
- case ENOMEM:<br>
- goto oom;<br>
--- <br>
-2.9.3<br>
-<br>
diff --git a/meta/recipes-devtools/cve-check-tool/files/0001-update-Compare-computed-vs-expected-sha256-digit-str.patch b/meta/recipes-devtools/cve-check-tool/files/0001-update-Compare-computed-vs-expected-sha256-digit-str.patch<br>
deleted file mode 100644<br>
index 458c0cc84e5..00000000000<br>
--- a/meta/recipes-devtools/cve-check-tool/files/0001-update-Compare-computed-vs-expected-sha256-digit-str.patch<br>
+++ /dev/null<br>
@@ -1,52 +0,0 @@<br>
-From b0426e63c9ac61657e029f689bcb8dd051e752c6 Mon Sep 17 00:00:00 2001<br>
-From: Sergey Popovich <<a href="mailto:popovich_sergei@mail.ua" target="_blank">popovich_sergei@mail.ua</a>><br>
-Date: Fri, 21 Apr 2017 07:32:23 -0700<br>
-Subject: [PATCH] update: Compare computed vs expected sha256 digit string<br>
- ignoring case<br>
-<br>
-We produce sha256 digest string using %x snprintf()<br>
-qualifier for each byte of digest which uses alphabetic<br>
-characters from "a" to "f" in lower case to represent<br>
-integer values from 10 to 15.<br>
-<br>
-Previously all of the NVD META files supply sha256<br>
-digest string for corresponding XML file in lower case.<br>
-<br>
-However due to some reason this changed recently to<br>
-provide digest digits in upper case causing fetched<br>
-data consistency checks to fail. This prevents database<br>
-from being updated periodically.<br>
-<br>
-While commit c4f6e94 (update: Do not treat sha256 failure<br>
-as fatal if requested) adds useful option to skip<br>
-digest validation at all and thus provides workaround for<br>
-this situation, it might be unacceptable for some<br>
-deployments where we need to ensure that downloaded<br>
-data is consistent before start parsing it and update<br>
-SQLite database.<br>
-<br>
-Use strcasecmp() to compare two digest strings case<br>
-insensitively and addressing this case.<br>
-<br>
-Upstream-Status: Backport<br>
-Signed-off-by: Sergey Popovich <<a href="mailto:popovich_sergei@mail.ua" target="_blank">popovich_sergei@mail.ua</a>><br>
----<br>
- src/update.c | 2 +-<br>
- 1 file changed, 1 insertion(+), 1 deletion(-)<br>
-<br>
-diff --git a/src/update.c b/src/update.c<br>
-index 8588f38..3cc6b67 100644<br>
---- a/src/update.c<br>
-+++ b/src/update.c<br>
-@@ -187,7 +187,7 @@ static bool nvdcve_data_ok(const char *meta, const char *data)<br>
- snprintf(&csum_data[idx], len, "%02hhx", digest[i]);<br>
- }<br>
- <br>
-- ret = streq(csum_meta, csum_data);<br>
-+ ret = !strcasecmp(csum_meta, csum_data);<br>
- <br>
- err_unmap:<br>
- munmap(buffer, length);<br>
--- <br>
-2.11.0<br>
-<br>
diff --git a/meta/recipes-devtools/cve-check-tool/files/check-for-malloc_trim-before-using-it.patch b/meta/recipes-devtools/cve-check-tool/files/check-for-malloc_trim-before-using-it.patch<br>
deleted file mode 100644<br>
index 0774ad946a4..00000000000<br>
--- a/meta/recipes-devtools/cve-check-tool/files/check-for-malloc_trim-before-using-it.patch<br>
+++ /dev/null<br>
@@ -1,51 +0,0 @@<br>
-From ce64633b9733e962b8d8482244301f614d8b5845 Mon Sep 17 00:00:00 2001<br>
-From: Khem Raj <<a href="mailto:raj.khem@gmail.com" target="_blank">raj.khem@gmail.com</a>><br>
-Date: Mon, 22 Aug 2016 22:54:24 -0700<br>
-Subject: [PATCH] Check for malloc_trim before using it<br>
-<br>
-malloc_trim is gnu specific and not all libc<br>
-implement it, threfore write a configure check<br>
-to poke for it first and use the define to<br>
-guard its use.<br>
-<br>
-Helps in compiling on musl based systems<br>
-<br>
-Signed-off-by: Khem Raj <<a href="mailto:raj.khem@gmail.com" target="_blank">raj.khem@gmail.com</a>><br>
----<br>
-Upstream-Status: Submitted [<a href="https://github.com/ikeydoherty/cve-check-tool/pull/48" rel="noreferrer" target="_blank">https://github.com/ikeydoherty/cve-check-tool/pull/48</a>]<br>
- <a href="http://configure.ac" rel="noreferrer" target="_blank">configure.ac</a> | 2 ++<br>
- src/core.c | 4 ++--<br>
- 2 files changed, 4 insertions(+), 2 deletions(-)<br>
-<br>
-diff --git a/<a href="http://configure.ac" rel="noreferrer" target="_blank">configure.ac</a> b/<a href="http://configure.ac" rel="noreferrer" target="_blank">configure.ac</a><br>
-index d3b66ce..79c3542 100644<br>
---- a/<a href="http://configure.ac" rel="noreferrer" target="_blank">configure.ac</a><br>
-+++ b/<a href="http://configure.ac" rel="noreferrer" target="_blank">configure.ac</a><br>
-@@ -19,6 +19,8 @@ m4_define([json_required_version], [0.16.0])<br>
- m4_define([openssl_required_version],[1.0.0])<br>
- # TODO: Set minimum sqlite<br>
- <br>
-+AC_CHECK_FUNCS_ONCE(malloc_trim)<br>
-+<br>
- PKG_CHECK_MODULES(CVE_CHECK_TOOL,<br>
- [<br>
- glib-2.0 >= glib_required_version,<br>
-diff --git a/src/core.c b/src/core.c<br>
-index 6263031..0d5df29 100644<br>
---- a/src/core.c<br>
-+++ b/src/core.c<br>
-@@ -498,9 +498,9 @@ bool cve_db_load(CveDB *self, const char *fname)<br>
- }<br>
- <br>
- b = true;<br>
--<br>
-+#ifdef HAVE_MALLOC_TRIM<br>
- malloc_trim(0);<br>
--<br>
-+#endif<br>
- xmlFreeTextReader(r);<br>
- if (fd) {<br>
- close(fd);<br>
--- <br>
-2.9.3<br>
-<br>
-- <br>
2.20.1<br>
<br>
-- <br>
_______________________________________________<br>
Openembedded-core mailing list<br>
<a href="mailto:Openembedded-core@lists.openembedded.org" target="_blank">Openembedded-core@lists.openembedded.org</a><br>
<a href="http://lists.openembedded.org/mailman/listinfo/openembedded-core" rel="noreferrer" target="_blank">http://lists.openembedded.org/mailman/listinfo/openembedded-core</a><br>
</blockquote></div></div>