[OE-core] [PATCH] Rework patch fuzz handling
Andreas Müller
schnitzeltony at gmail.com
Fri Apr 5 23:45:56 UTC 2019
Currently there are three issues which can be enhanced:
1. Fuzz warnings cannot be configured as errors for hardening. It happened
often to me that these warnings were overseen and detected after commits
were already out.
2. The output is too verbose - particularly when more than one file is
affected. Meanwhile all users should know why patch fuzz check is performed.
So move links with background information to insane.bbclass.
3. Reduce copy & paste effort slightly by printing PN (nit: <recipe> was not
a correct suggestion e.g for native extended recipe - see example below)
To achieve patch.py drops patch-fuzz info encapsulated by a header- and footer-
string into log.do_patch. With this insane.bbclass can drop warnings/errors
depending on 'patch-fuzz' in ERROR_QA or WARN_QA. Default remains unchanged:
Spit out warnings only.
A message for two fuzzed patches and 'pact-fuzz' in ERROR_QA now looks like:
| ERROR: autoconf-native-2.69-r11 do_patch: Fuzz detected:
|
| Applying patch autoreconf-exclude.patch
| patching file bin/autoreconf.in
| Hunk #1 succeeded at 73 with fuzz 1 (offset -3 lines).
| Hunk #2 succeeded at 143 (offset 6 lines).
| Hunk #3 succeeded at 167 (offset 6 lines).
| Hunk #4 succeeded at 177 (offset 6 lines).
| Hunk #5 succeeded at 281 (offset 15 lines).
| Hunk #6 succeeded at 399 (offset 15 lines).
| Hunk #7 succeeded at 571 (offset 20 lines).
| Hunk #8 succeeded at 612 (offset 20 lines).
| Hunk #9 succeeded at 636 (offset 20 lines).
| Hunk #10 succeeded at 656 (offset 20 lines).
| Hunk #11 succeeded at 683 (offset 20 lines).
|
| Applying patch autoreconf-gnuconfigize.patch
| patching file bin/autoreconf.in
| Hunk #1 succeeded at 55 with fuzz 1 (offset -3 lines).
| Hunk #3 succeeded at 663 (offset 18 lines).
|
| The context lines in the patches can be updated with devtool:
|
| devtool modify autoconf-native
| devtool finish --force-patch-refresh autoconf-native <layer_path>
|
| Don't forget to review changes done by devtool!
|
| ERROR: autoconf-native-2.69-r11 do_patch: QA Issue: Patch log indicates that patches do not apply cleanly. [patch-fuzz]
Signed-off-by: Andreas Müller <schnitzeltony at gmail.com>
---
meta/classes/insane.bbclass | 53 ++++++++++++++++++++++++++++++++++++-
meta/lib/oe/patch.py | 19 +++----------
2 files changed, 56 insertions(+), 16 deletions(-)
diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index c8d18c07f55..7b0efc21892 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -27,7 +27,7 @@ WARN_QA ?= "ldflags useless-rpaths rpaths staticdev libdir xorg-driver-abi \
installed-vs-shipped compile-host-path install-host-path \
pn-overrides infodir build-deps \
unknown-configure-option symlink-to-sysroot multilib \
- invalid-packageconfig host-user-contaminated uppercase-pn \
+ invalid-packageconfig host-user-contaminated uppercase-pn patch-fuzz \
"
ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch pkgconfig la \
perms dep-cmp pkgvarcheck perm-config perm-line perm-link \
@@ -1033,6 +1033,54 @@ python do_qa_staging() {
bb.fatal("QA staging was broken by the package built above")
}
+python do_qa_patch() {
+ import subprocess
+
+ ###########################################################################
+ # Check patch.log for fuzz warnings
+ #
+ # Further information on why we check for patch fuzz warnings:
+ # http://lists.openembedded.org/pipermail/openembedded-core/2018-March/148675.html
+ # https://bugzilla.yoctoproject.org/show_bug.cgi?id=10450
+ ###########################################################################
+
+ logdir = d.getVar('T')
+ patchlog = os.path.join(logdir,"log.do_patch")
+
+ if os.path.exists(patchlog):
+ fuzzheader = '--- Patch fuzz start ---'
+ fuzzfooter = '--- Patch fuzz end ---'
+ statement = "grep -e '%s' %s > /dev/null" % (fuzzheader, patchlog)
+ if subprocess.call(statement, shell=True) == 0:
+ msg = "Fuzz detected:\n\n"
+ fuzzmsg = ""
+ inFuzzInfo = False
+ f = open(patchlog, "r")
+ for line in f:
+ if fuzzheader in line:
+ inFuzzInfo = True
+ fuzzmsg = ""
+ elif fuzzfooter in line:
+ fuzzmsg = fuzzmsg.replace('\n\n', '\n')
+ msg += fuzzmsg
+ msg += "\n"
+ inFuzzInfo = False
+ elif inFuzzInfo and not 'Now at patch' in line:
+ fuzzmsg += line
+ f.close()
+ msg += "The context lines in the patches can be updated with devtool:\n"
+ msg += "\n"
+ msg += " devtool modify %s\n" % d.getVar('PN')
+ msg += " devtool finish --force-patch-refresh %s <layer_path>\n\n" % d.getVar('PN')
+ msg += "Don't forget to review changes done by devtool!\n"
+ if 'patch-fuzz' in d.getVar('ERROR_QA'):
+ bb.error(msg)
+ elif 'patch-fuzz' in d.getVar('WARN_QA'):
+ bb.warn(msg)
+ msg = "Patch log indicates that patches do not apply cleanly."
+ package_qa_handle_error("patch-fuzz", msg, d)
+}
+
python do_qa_configure() {
import subprocess
@@ -1141,6 +1189,9 @@ python do_qa_unpack() {
#addtask qa_staging after do_populate_sysroot before do_build
do_populate_sysroot[postfuncs] += "do_qa_staging "
+# Check for patch fuzz
+do_patch[postfuncs] += "do_qa_patch "
+
# Check broken config.log files, for packages requiring Gettext which
# don't have it in DEPENDS.
#addtask qa_configure after do_configure before do_compile
diff --git a/meta/lib/oe/patch.py b/meta/lib/oe/patch.py
index 7dd31d9d463..f43cf047c06 100644
--- a/meta/lib/oe/patch.py
+++ b/meta/lib/oe/patch.py
@@ -38,21 +38,10 @@ def runcmd(args, dir = None):
if exitstatus != 0:
raise CmdError(cmd, exitstatus >> 8, output)
if " fuzz " in output:
- bb.warn("""
-Some of the context lines in patches were ignored. This can lead to incorrectly applied patches.
-The context lines in the patches can be updated with devtool:
-
- devtool modify <recipe>
- devtool finish --force-patch-refresh <recipe> <layer_path>
-
-Then the updated patches and the source tree (in devtool's workspace)
-should be reviewed to make sure the patches apply in the correct place
-and don't introduce duplicate lines (which can, and does happen
-when some of the context is ignored). Further information:
-http://lists.openembedded.org/pipermail/openembedded-core/2018-March/148675.html
-https://bugzilla.yoctoproject.org/show_bug.cgi?id=10450
-Details:
-{}""".format(output))
+ # Drop patch fuzz info with header and footer to log file so
+ # insane.bbclass can handle to throw error/warning
+ bb.note("--- Patch fuzz start ---\n%s\n--- Patch fuzz end ---" % format(output))
+
return output
finally:
--
2.20.1
More information about the Openembedded-core
mailing list