<div dir="ltr"><div class="gmail_quote"><div dir="ltr">On Sun, Apr 10, 2016 at 3:10 AM Richard Purdie <<a href="mailto:richard.purdie@linuxfoundation.org">richard.purdie@linuxfoundation.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">When multiple recipes which both provide something are being built, bitbake<br>
informs us that most likely one of them provides something the other doesn't,<br>
which is usually correct, but unfortunately it's rather painful to figure out<br>
exactly what that is.<br>
<br>
This patch dumps two sets of information, one is the provides information for<br>
each recipe, filtered so only common components are removed. The other is a list<br>
of dependees on the recipe, since sometimes this can easily identify why something<br>
is being built.<br>
<br>
Its not straightforward for bitbake to obtain the information but since the<br>
warning/error code path isn't the normal one, we can afford to go through some<br>
less than optimal processing to aid debugging.<br>
<br>
Also provide the same information even if we're showing a warning since its still<br>
useful.<br>
<br>
[YOCTO #8032]<br>
<br>
Signed-off-by: Richard Purdie <<a href="mailto:richard.purdie@linuxfoundation.org" target="_blank">richard.purdie@linuxfoundation.org</a>><br></blockquote><div><br></div><div>Nice work on this, thanks much. I think this will be a big help in diagnosing issues of this sort. I have a few very minor comments below.</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py<br>
index 2990de7..ac3cdae 100644<br>
--- a/bitbake/lib/bb/runqueue.py<br>
+++ b/bitbake/lib/bb/runqueue.py<br>
@@ -261,6 +261,13 @@ class RunQueueData:<br>
taskname = self.runq_task[task] + task_name_suffix<br>
return "%s, %s" % (fn, taskname)<br>
<br>
+ def get_short_user_idstring(self, task, task_name_suffix = ""):<br>
+ fn = self.taskData.fn_index[self.runq_fnid[task]]<br>
+ pn = self.dataCache.pkg_fn[fn]<br>
+ taskname = self.runq_task[task] + task_name_suffix<br>
+ return "%s:%s" % (pn, taskname)<br>
+<br>
+<br>
def get_task_id(self, fnid, taskname):<br>
for listid in xrange(len(self.runq_fnid)):<br>
if self.runq_fnid[listid] == fnid and self.runq_task[listid] == taskname:<br>
@@ -753,11 +760,72 @@ class RunQueueData:<br>
seen_pn.append(pn)<br>
else:<br>
bb.fatal("Multiple versions of %s are due to be built (%s). Only one version of a given PN should be built in any given build. You likely need to set PREFERRED_VERSION_%s to select the correct version or don't depend on multiple versions." % (pn, " ".join(prov_list[prov]), pn))<br>
- msg = "Multiple .bb files are due to be built which each provide %s (%s)." % (prov, " ".join(prov_list[prov]))<br>
+ msg = "Multiple .bb files are due to be built which each provide %s:\n %s" % (prov, "\n ".join(prov_list[prov]))<br>
+ #<br>
+ # Construct a list of things which uniquely depend on each provider<br>
+ # since this may help the user figure out which dependency is triggering this warning<br>
+ #<br>
+ msg += "\nA list of tasks depending on these providers is shown and may help explain where the dependency comes from."<br>
+ deplist = {}<br>
+ commondeps = None<br>
+ for provfn in prov_list[prov]:<br>
+ deps = set()<br>
+ for task in xrange(len(self.runq_fnid)):<br>
+ fn = taskData.fn_index[self.runq_fnid[task]]<br>
+ if fn != provfn:<br>
+ continue<br></blockquote><div><br></div><div><br></div><div>I think this would be more pythonic:</div><div><br></div><div> for task, fnid in enumerate(self.runq_fnid):</div><div> fn = taskData.fn_index[fnid]</div><div> </div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
+ for dep in self.runq_revdeps[task]:<br>
+ fn = taskData.fn_index[self.runq_fnid[dep]]<br>
+ if fn == provfn:<br>
+ continue<br>
+ deps.add(self.get_short_user_idstring(dep))<br></blockquote><div><br></div><div><br></div><div>Here we look up the fn of dep, but then get_short_user_idstring() does so as well, might be worth passing it in instead, unless it's used elsewhere where the fn isn't yet available.</div><div> </div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
+ if not commondeps:<br>
+ commondeps = set(deps)<br>
+ else:<br>
+ commondeps &= deps<br>
+ deplist[provfn] = deps<br>
+ for provfn in deplist:<br>
+ msg += "\n%s has unique dependees:\n %s" % (provfn, "\n ".join(deplist[provfn] - commondeps))<br>
+ #<br>
+ # Construct a list of provides and runtime providers for each recipe<br>
+ # (rprovides has to cover RPROVIDES, PACKAGES, PACKAGES_DYNAMIC)<br>
+ #<br>
+ msg += "\nIt could be that one recipe provides something the other doesn't and should. The following provider and runtime provider differences may be helpful."<br>
+ provlist = {}<br></blockquote><div><br></div><div><br></div><div>Might want to rename this variable to improve readability, just because there's also prov_list :)</div><div> </div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
+ rprovlist = {}<br>
+ commonprovs = None<br>
+ commonrprovs = None<br>
+ for provfn in prov_list[prov]:<br>
+ provides = set(self.dataCache.fn_provides[provfn])<br>
+ rprovides = set()<br>
+ for rprovide in self.dataCache.rproviders:<br>
+ if provfn in self.dataCache.rproviders[rprovide]:<br>
+ rprovides.add(rprovide)<br>
+ for package in self.dataCache.packages:<br>
+ if provfn in self.dataCache.packages[package]:<br>
+ rprovides.add(package)<br>
+ for package in self.dataCache.packages_dynamic:<br>
+ if provfn in self.dataCache.packages_dynamic[package]:<br>
+ rprovides.add(package)<br></blockquote><div><br></div><div><br></div><div>Minor and largely personal preference either way, but these loops are simple enough they could be comprehensions. Don't feel too strongly either way on that.</div><div> </div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
+ if not commonprovs:<br>
+ commonprovs = set(provides)<br>
+ else:<br>
+ commonprovs &= provides<br>
+ provlist[provfn] = provides<br>
+ if not commonrprovs:<br>
+ commonrprovs = set(rprovides)<br>
+ else:<br>
+ commonrprovs &= rprovides<br>
+ rprovlist[provfn] = rprovides<br>
+ #msg += "\nCommon provides:\n %s" % ("\n ".join(commonprovs))<br>
+ #msg += "\nCommon rprovides:\n %s" % ("\n ".join(commonrprovs))<br>
+ for provfn in prov_list[prov]:<br>
+ msg += "\n%s has unique provides:\n %s" % (provfn, "\n ".join(provlist[provfn] - commonprovs))<br>
+ msg += "\n%s has unique rprovides:\n %s" % (provfn, "\n ".join(rprovlist[provfn] - commonrprovs))</blockquote><div><br></div><div><br></div><div>Might want to check to see if the list of uniques is empty before displaying its message, just to reduce possible confusion.</div><div><br></div><div>Thanks again,</div></div></div>