<html><head></head><body><div>On Tue, 2016-06-07 at 08:48 -0700, Christopher Larson wrote:</div><div>On Tue, Jun 7, 2016 at 7:50 AM, Joshua Lock <span dir="ltr"><<a href="mailto:joshua.g.lock@intel.com" target="_blank">joshua.g.lock@intel.com</a>></span> wrote:<br><blockquote type="cite"><div id=":16q" class="a3s aXjCH m1552b5843d6d0f92">Add a mechanism to run QA checks on a constructed image once it's<br>
complete. All checks will be run with any one failure resulting in<br>
a failed build.<br>
<br>
QA checks should be <span class="il">bitbake</span> functions which throw a<br>
NotImplementedError when the check QA fails, with any error<br>
messages passed to the exception.<br>
<br>
Specify which checks to run by adding them to IMAGE_QA_COMMANDS.<br>
<br>
i.e.<br>
<br>
IMAGE_QA_COMMANDS += " \<br>
image_check_everything_ok \<br>
"<br>
<br>
python image_check_everything_ok () {<br>
raise NotImplementedError('This check always fails')<br>
}<br>
<br>
This code is based heavily on the configuration upgrade code in<br>
sanity.<span class="il">bbclass</span>.<br>
<br>
[YOCTO #9448]<br>
<br>
Signed-off-by: Joshua Lock <<a href="mailto:joshua.g.lock@intel.com">joshua.g.lock@intel.com</a>></div><br></blockquote></div><blockquote type="cite"><div dir="ltr"><div class="gmail_extra"><div class="gmail_extra"><div class="gmail_quote"><blockquote type="cite"><div id=":16q" class="a3s aXjCH m1552b5843d6d0f92">Add a mechanism to run QA checks on a constructed image once it's<br>
complete. All checks will be run with any one failure resulting in<br>
a failed build.<br>
<br>
QA checks should be <span class="il">bitbake</span> functions which throw a<br>
NotImplementedError when the check QA fails, with any error<br>
messages passed to the exception.<br>
<br>
Specify which checks to run by adding them to IMAGE_QA_COMMANDS.<br>
<br>
i.e.<br>
<br>
IMAGE_QA_COMMANDS += " \<br>
image_check_everything_ok \<br>
"<br>
<br>
python image_check_everything_ok () {<br>
raise NotImplementedError('This check always fails')<br>
}<br>
<br>
This code is based heavily on the configuration upgrade code in<br>
sanity.<span class="il">bbclass</span>.<br>
<br>
[YOCTO #9448]<br>
<br>
Signed-off-by: Joshua Lock <<a href="mailto:joshua.g.lock@intel.com">joshua.g.lock@intel.com</a>></div><br></blockquote></div><br><br><div>What's the behavior if your qa function fails with a different exception? What if a user writes a shell qa check function, what's the behavior? Also, this seems like overloading the purpose of NotImplementedError. IMO It'd be cleaner to either use a custom exception or re-use python unit testing bits / use assert rather than subverting this one to a different purpose.</div><br>
</div><div><br></div><br><br><div>What's the behavior if your qa function fails with a different exception? What if a user writes a shell qa check function, what's the behavior? Also, this seems like overloading the purpose of NotImplementedError. IMO It'd be cleaner to either use a custom exception or re-use python unit testing bits / use assert rather than subverting this one to a different purpose.</div><br>
</div></div>
</blockquote><div><br></div><div>Different exceptions to NotImplementedError result in a backtrace… </div><div><br></div><div>The python exception pattern using NotImplementedError I copied from the configuration upgrade code in sanity.bbclass, because I want to replicate that same behaviour where the checks are able to pass a reason back to the task which calls the check functions.</div><div><br></div><div>Shell functions are a good point, not least of all because I suspect many checks could be much more concise with sh.</div><div><br></div><div>I'll go back to the drawing board for a more generic solution that supports both sh and python tasks.</div><div><br></div><div>Thanks for the review,</div><div><br></div><div>Joshua</div><div><br></div><div><br></div></body></html>