<html><head></head><body><div>On Fri, 2018-02-16 at 21:14 +0100, Martin Jansa wrote:</div><blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"><div dir="ltr"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">+ v = max(limit, v)</span><br><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br></span></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">this seems to be doing the opposite of what the limit was used for before:</span></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br></span></div><div><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">- v = min(64, int(v))</span><br></span></div></div></blockquote><div><br></div><div>Hmm, you're right</div><div><br></div><blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Feb 12, 2018 at 8:39 PM, Joshua Watt <span dir="ltr"><<a href="mailto:jpewhacker@gmail.com" target="_blank">jpewhacker@gmail.com</a>></span> wrote:<br><blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">The code to extract the integer number of parallel build threads and<br>
construct a new argument from them has started to be copied in multiple<br>
locations, so create two new helper utilities to aid recipes.<br>
<br>
The first helper (parallel_make()) extracts the integer number of<br>
parallel build threads from PARALLEL_MAKE. The second<br>
(parallel_make_argument()) does the same and then puts the result back<br>
into a format string, optionally clamping it to some maximum value.<br>
<br>
Additionally, rework the oe-core recipes that were manually doing this<br>
to use the new helper utilities.<br>
<br>
Signed-off-by: Joshua Watt <<a href="mailto:JPEWhacker@gmail.com">JPEWhacker@gmail.com</a>><br>
---<br>
meta/classes/go.bbclass | 19 +---------------<br>
meta/classes/waf.bbclass | 24 +-------------------<br>
meta/lib/oe/utils.py | 43 ++++++++++++++++++++++++++++++<wbr>++++++<br>
meta/recipes-core/ovmf/<a href="http://ovmf_git.bb" rel="noreferrer" target="_blank">ovmf_<wbr>git.bb</a> | 2 +-<br>
meta/recipes-support/boost/<wbr>boost.inc | 24 +-------------------<br>
5 files changed, 47 insertions(+), 65 deletions(-)<br>
<br>
diff --git a/meta/classes/go.bbclass b/meta/classes/go.bbclass<br>
index 09b01a84c37..7ecd8c92541 100644<br>
--- a/meta/classes/go.bbclass<br>
+++ b/meta/classes/go.bbclass<br>
@@ -1,23 +1,6 @@<br>
inherit goarch ptest<br>
<br>
-def get_go_parallel_make(d):<br>
- pm = (d.getVar('PARALLEL_MAKE') or '').split()<br>
- # look for '-j' and throw other options (e.g. '-l') away<br>
- # because they might have a different meaning in golang<br>
- while pm:<br>
- opt = pm.pop(0)<br>
- if opt == '-j':<br>
- v = pm.pop(0)<br>
- elif opt.startswith('-j'):<br>
- v = opt[2:].strip()<br>
- else:<br>
- continue<br>
-<br>
- return '-p %d' % int(v)<br>
-<br>
- return ""<br>
-<br>
-GO_PARALLEL_BUILD ?= "${@get_go_parallel_make(d)}"<br>
+GO_PARALLEL_BUILD ?= "${@oe.utils.parallel_make_<wbr>argument(d, '-p %d')}"<br>
<br>
GOROOT_class-native = "${STAGING_LIBDIR_NATIVE}/go"<br>
GOROOT_class-nativesdk = "${STAGING_DIR_TARGET}${<wbr>libdir}/go"<br>
diff --git a/meta/classes/waf.bbclass b/meta/classes/waf.bbclass<br>
index bdbdc56767c..f9a851d0759 100644<br>
--- a/meta/classes/waf.bbclass<br>
+++ b/meta/classes/waf.bbclass<br>
@@ -3,28 +3,6 @@ DISABLE_STATIC = ""<br>
<br>
EXTRA_OECONF_append = " ${PACKAGECONFIG_CONFARGS}"<br>
<br>
-def get_waf_parallel_make(d):<br>
- pm = d.getVar('PARALLEL_MAKE')<br>
- if pm:<br>
- # look for '-j' and throw other options (e.g. '-l') away<br>
- # because they might have different meaning in bjam<br>
- pm = pm.split()<br>
- while pm:<br>
- v = None<br>
- opt = pm.pop(0)<br>
- if opt == '-j':<br>
- v = pm.pop(0)<br>
- elif opt.startswith('-j'):<br>
- v = opt[2:].strip()<br>
- else:<br>
- v = None<br>
-<br>
- if v:<br>
- v = min(64, int(v))<br>
- return '-j' + str(v)<br>
-<br>
- return ""<br>
-<br>
python waf_preconfigure() {<br>
import subprocess<br>
from distutils.version import StrictVersion<br>
@@ -47,7 +25,7 @@ waf_do_configure() {<br>
<br>
do_compile[progress] = "outof:^\[\s*(\d+)/\s*(\d+)\]\<wbr>s+"<br>
waf_do_compile() {<br>
- ${S}/waf build ${@get_waf_parallel_make(d)}<br>
+ ${S}/waf build ${@oe.utils.parallel_make_<wbr>argument(d, '-j%d', limit=64)}<br>
}<br>
<br>
waf_do_install() {<br>
diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py<br>
index 7a79d752b69..ec91927233a 100644<br>
--- a/meta/lib/oe/utils.py<br>
+++ b/meta/lib/oe/utils.py<br>
@@ -156,6 +156,49 @@ def any_distro_features(d, features, truevalue="1", falsevalue=""):<br>
"""<br>
return bb.utils.contains_any("DISTRO_<wbr>FEATURES", features, truevalue, falsevalue, d)<br>
<br>
+def parallel_make(d):<br>
+ """<br>
+ Return the integer value for the number of parallel threads to use when<br>
+ building, scraped out of PARALLEL_MAKE. If no parallelization option is<br>
+ found, returns None<br>
+<br>
+ e.g. if PARALLEL_MAKE = "-j 10", this will return 10 as an integer.<br>
+ """<br>
+ pm = (d.getVar('PARALLEL_MAKE') or '').split()<br>
+ # look for '-j' and throw other options (e.g. '-l') away<br>
+ while pm:<br>
+ opt = pm.pop(0)<br>
+ if opt == '-j':<br>
+ v = pm.pop(0)<br>
+ elif opt.startswith('-j'):<br>
+ v = opt[2:].strip()<br>
+ else:<br>
+ continue<br>
+<br>
+ return int(v)<br>
+<br>
+ return None<br>
+<br>
+def parallel_make_argument(d, fmt, limit=None):<br>
+ """<br>
+ Helper utility to construct a parallel make argument from the number of<br>
+ parallel threads specified in PARALLEL_MAKE.<br>
+<br>
+ Returns the input format string `fmt` where a single '%d' will be expanded<br>
+ with the number of parallel threads to use. If `limit` is specified, the<br>
+ number of parallel threads will be no larger than it. If no parallelization<br>
+ option is found in PARALLEL_MAKE, returns an empty string<br>
+<br>
+ e.g. if PARALLEL_MAKE = "-j 10", parallel_make_argument(d, "-n %d") will return<br>
+ "-n 10"<br>
+ """<br>
+ v = parallel_make(d)<br>
+ if v:<br>
+ if limit:<br>
+ v = max(limit, v)<br>
+ return fmt % v<br>
+ return ''<br>
+<br>
def packages_filter_out_system(d):<br>
"""<br>
Return a list of packages from PACKAGES with the "system" packages such as<br>
diff --git a/meta/recipes-core/ovmf/<a href="http://ovmf_git.bb" rel="noreferrer" target="_blank">ovmf_<wbr>git.bb</a> b/meta/recipes-core/ovmf/<a href="http://ovmf_git.bb" rel="noreferrer" target="_blank">ovmf_<wbr>git.bb</a><br>
index fa0d66291d1..8750b3c528d 100644<br>
--- a/meta/recipes-core/ovmf/<a href="http://ovmf_git.bb" rel="noreferrer" target="_blank">ovmf_<wbr>git.bb</a><br>
+++ b/meta/recipes-core/ovmf/<a href="http://ovmf_git.bb" rel="noreferrer" target="_blank">ovmf_<wbr>git.bb</a><br>
@@ -151,7 +151,7 @@ do_compile_class-native() {<br>
<br>
do_compile_class-target() {<br>
export LFLAGS="${LDFLAGS}"<br>
- PARALLEL_JOBS="${@ '${PARALLEL_MAKE}'.replace('-<wbr>j', '-n ')}"<br>
+ PARALLEL_JOBS="${@oe.utils.<wbr>parallel_make_argument(d, '-n %d')}"<br>
OVMF_ARCH="X64"<br>
if [ "${TARGET_ARCH}" != "x86_64" ] ; then<br>
OVMF_ARCH="IA32"<br>
diff --git a/meta/recipes-support/boost/<wbr>boost.inc b/meta/recipes-support/boost/<wbr>boost.inc<br>
index 41fc90fb211..0461ec6fcf1 100644<br>
--- a/meta/recipes-support/boost/<wbr>boost.inc<br>
+++ b/meta/recipes-support/boost/<wbr>boost.inc<br>
@@ -135,29 +135,7 @@ BJAM_TOOLS = "--ignore-site-config \<br>
<br>
# use PARALLEL_MAKE to speed up the build, but limit it by -j 64, greater parallelism causes bjam to segfault or to ignore -j<br>
# <a href="https://svn.boost.org/trac/boost/ticket/7634" rel="noreferrer" target="_blank">https://svn.boost.org/trac/<wbr>boost/ticket/7634</a><br>
-def get_boost_parallel_make(d):<br>
- pm = d.getVar('PARALLEL_MAKE')<br>
- if pm:<br>
- # look for '-j' and throw other options (e.g. '-l') away<br>
- # because they might have different meaning in bjam<br>
- pm = pm.split()<br>
- while pm:<br>
- v = None<br>
- opt = pm.pop(0)<br>
- if opt == '-j':<br>
- v = pm.pop(0)<br>
- elif opt.startswith('-j'):<br>
- v = opt[2:].strip()<br>
- else:<br>
- v = None<br>
-<br>
- if v:<br>
- v = min(64, int(v))<br>
- return '-j' + str(v)<br>
-<br>
- return ""<br>
-<br>
-BOOST_PARALLEL_MAKE = "${@get_boost_parallel_make(d)<wbr>}"<br>
+BOOST_PARALLEL_MAKE = "${@oe.utils.parallel_make_<wbr>argument(d, '-j%d', limit=64)}"<br>
BJAM_OPTS = '${BOOST_PARALLEL_MAKE} -d+2 -q \<br>
${BJAM_TOOLS} \<br>
-sBOOST_BUILD_USER_CONFIG=${<wbr>WORKDIR}/user-config.jam \<br>
<span class="HOEnZb"><font color="#888888">--<br>
2.14.3<br>
<br>
--<br>
______________________________<wbr>_________________<br>
Openembedded-core mailing list<br>
<a href="mailto:Openembedded-core@lists.openembedded.org">Openembedded-core@lists.<wbr>openembedded.org</a><br>
<a href="http://lists.openembedded.org/mailman/listinfo/openembedded-core" rel="noreferrer" target="_blank">http://lists.openembedded.org/<wbr>mailman/listinfo/openembedded-<wbr>core</a><br>
</font></span></blockquote></div><br></div>
</blockquote></body></html>