[bitbake-devel] [PATCH 2/4] Hob: log for Hob and allow users to show logs after successful build
Barros Pena, Belen
belen.barros.pena at intel.com
Fri Apr 6 08:10:57 UTC 2012
Thanks, Jessica. I think you are right.
1. User build an image which includes build packages. In this case the
log will show the build info for both packages and image
'View log' in the Image details screen will provide access to this log
file
2. User just build packages. In this case the log will show build package
info only.
'View log' in the Packages screen will provide access to this log file
3. User has previous build packages then do image customization and build
the image. Then the log will contain build image info.
'View log' in the Image details screen will provide access to this log file
I have added this to the bug 1991, together with a screenshot of the
Packages screen with the 'View log' action
Cheers
Belen
On 05/04/2012 15:49, "Zhang, Jessica" <jessica.zhang at intel.com> wrote:
>Hi Belen,
>
>I've talked with Shane to clarify all the different scenarios of log
>contents, so we have following combinations:
>
>1. User build an image which includes build packages. In this case the
>log will show the build info for both packages and image
>
>2. User just build packages. In this case the log will show build
>package info only.
>
>3. User has previous build packages then do image customization and build
>the image. Then the log will contain build image info.
>
>I think if we have a retrieve log button after both build image and build
>packages, we should be able to cover all the above scenarios with the
>corresponding log content.
>
>Thanks,
>Jessica
>
>-----Original Message-----
>From: Barros Pena, Belen
>Sent: Wednesday, April 04, 2012 9:44 AM
>To: Zhang, Jessica; Wang, Shane; bitbake-devel at lists.openembedded.org
>Cc: Giulia Piu
>Subject: Re: [bitbake-devel] [PATCH 2/4] Hob: log for Hob and allow users
>to show logs after successful build
>
>Sorry, guys. Now I am confused. Are the following assumptions correct?
>
>1. Building packages via Hob will generate a log file stored somewhere in
>the YP directory 2. Building an image via Hob will generate a log file
>stored somewhere in the YP directory 3. The above logs are two separate
>files
>
>Assuming the above is correct we need:
>
>1. A 'View log' button in the Image details screen to access the building
>image log file 2. A 'View log' button in the Packages screen to access
>the building packages log file
>
>Let me know if the above makes sense.
>
>Thanks,
>
>Belen
>
>
>
>On 04/04/2012 16:59, "Zhang, Jessica" <jessica.zhang at intel.com> wrote:
>
>>Hi Belen,
>>
>>I'd expect user to be able to retrieve all logs, image or package.
>>
>>Thanks,
>>Jessica
>>
>>-----Original Message-----
>>From: Barros Pena, Belen
>>Sent: Tuesday, April 03, 2012 2:25 AM
>>To: Wang, Shane; bitbake-devel at lists.openembedded.org
>>Cc: Zhang, Jessica; Giulia Piu
>>Subject: Re: [bitbake-devel] [PATCH 2/4] Hob: log for Hob and allow
>>users to show logs after successful build
>>
>>Sorry: this was an omission on the design side. If deemed appropriate,
>>we can incorporate an option to retrieve the packages build log in the
>>Packages screen. I'll leave it to Jessica, who reported 1991, to decide
>>if this is something we should provide.
>>
>>Cheers
>>
>>Belen
>>
>>
>>On 02/04/2012 16:22, "Shane Wang" <shane.wang at intel.com> wrote:
>>
>>>If users build images in Hob, record logs and allow users to retrieve
>>>the logs after successful build.
>>>
>>>The logs are generated if and only if:
>>> - users do "just bake"
>>> - users do "build image" after "build packages"
>>>
>>>Only "build packages" will not write logs because there is no chance
>>>for users to retrieve on the GUI.
>>>
>>>[Yocto #1991]
>>>
>>>Signed-off-by: Shane Wang <shane.wang at intel.com>
>>>---
>>> bitbake/lib/bb/ui/crumbs/builder.py | 36 +++++++++++++++--
>>> bitbake/lib/bb/ui/crumbs/hobeventhandler.py | 3 +
>>> bitbake/lib/bb/ui/crumbs/imagedetailspage.py | 14 ++++++-
>>> bitbake/lib/bb/ui/crumbs/runningbuild.py | 53
>>>++++++++++++++++++++++++++
>>> 4 files changed, 101 insertions(+), 5 deletions(-)
>>>
>>>diff --git a/bitbake/lib/bb/ui/crumbs/builder.py
>>>b/bitbake/lib/bb/ui/crumbs/builder.py
>>>index 6587734..de4fb29 100755
>>>--- a/bitbake/lib/bb/ui/crumbs/builder.py
>>>+++ b/bitbake/lib/bb/ui/crumbs/builder.py
>>>@@ -26,6 +26,7 @@ import copy
>>> import os
>>> import subprocess
>>> import shlex
>>>+import logging
>>> from bb.ui.crumbs.template import TemplateMgr from
>>>bb.ui.crumbs.imageconfigurationpage import ImageConfigurationPage
>>>from bb.ui.crumbs.recipeselectionpage import RecipeSelectionPage @@
>>>-260,6
>>>+261,11 @@ class Builder(gtk.Window):
>>>
>>> self.template = None
>>>
>>>+ # logger
>>>+ self.logger = logging.getLogger("BitBake")
>>>+ self.consolelog = None
>>>+ self.current_logfile = None
>>>+
>>> # build step
>>> self.current_step = None
>>> self.previous_step = None
>>>@@ -287,6 +293,7 @@ class Builder(gtk.Window):
>>> self.handler.build.connect("build-failed",
>>>self.handler_build_failed_cb)
>>> self.handler.build.connect("task-started",
>>>self.handler_task_started_cb)
>>> self.handler.build.connect("log-error",
>>>self.handler_build_failure_cb)
>>>+ self.handler.build.connect("log",
>>>self.handler_build_log_cb)
>>> self.handler.connect("generating-data",
>>>self.handler_generating_data_cb)
>>> self.handler.connect("data-generated",
>>>self.handler_data_generated_cb)
>>> self.handler.connect("command-succeeded",
>>>self.handler_command_succeeded_cb)
>>>@@ -396,7 +403,7 @@ class Builder(gtk.Window):
>>> elif next_step == self.PACKAGE_GENERATING or next_step ==
>>>self.FAST_IMAGE_GENERATING:
>>> # both PACKAGE_GENEATING and FAST_IMAGE_GENERATING share
>>>the same page
>>> self.build_details_page.show_page(next_step)
>>>- self.generate_packages()
>>>+ self.generate_packages(next_step ==
>>>self.FAST_IMAGE_GENERATING)
>>>
>>> elif next_step == self.PACKAGE_GENERATED:
>>> pass
>>>@@ -405,7 +412,7 @@ class Builder(gtk.Window):
>>> # after packages are generated, selected_packages need to
>>> # be updated in package_model per selected_image in
>>>recipe_model
>>> self.build_details_page.show_page(next_step)
>>>- self.generate_image()
>>>+ self.generate_image(self.current_step ==
>>>self.FAST_IMAGE_GENERATING)
>>>
>>> elif next_step == self.IMAGE_GENERATED:
>>> self.image_details_page.show_page(next_step)
>>>@@ -454,7 +461,10 @@ class Builder(gtk.Window):
>>> left =
>>>self.package_model.set_selected_packages(selected_packages)
>>> self.configuration.selected_packages += left
>>>
>>>- def generate_packages(self):
>>>+ def generate_packages(self, log = False):
>>>+ if log:
>>>+ self.current_logfile = self.handler.get_logfile()
>>>+ self.do_log(self.current_logfile)
>>> # Build packages
>>> _, all_recipes = self.recipe_model.get_selected_recipes()
>>> self.set_user_config()
>>>@@ -466,7 +476,10 @@ class Builder(gtk.Window):
>>> self.set_user_config()
>>> self.handler.generate_recipes()
>>>
>>>- def generate_image(self):
>>>+ def generate_image(self, cont = False):
>>>+ if not cont:
>>>+ self.current_logfile = self.handler.get_logfile()
>>>+ self.do_log(self.current_logfile)
>>> # Build image
>>> self.set_user_config()
>>> all_packages = self.package_model.get_selected_packages()
>>>@@ -708,6 +721,10 @@ class Builder(gtk.Window):
>>> def handler_build_failure_cb(self, running_build):
>>> self.build_details_page.show_issues()
>>>
>>>+ def handler_build_log_cb(self, running_build, func, obj):
>>>+ if hasattr(self.logger, func):
>>>+ getattr(self.logger, func)(obj)
>>>+
>>> def destroy_window_cb(self, widget, event):
>>> lbl = "<b>Do you really want to exit the Hob image
>>>creator?</b>"
>>> dialog = CrumbsMessageDialog(self, lbl,
>>>gtk.STOCK_DIALOG_INFO) @@ -1033,3 +1050,14 @@ class Builder(gtk.Window):
>>> self.handler.cancel_build()
>>> elif response == gtk.RESPONSE_YES:
>>> self.handler.cancel_build(True)
>>>+
>>>+ def do_log(self, consolelogfile = None):
>>>+ if consolelogfile:
>>>+ if self.consolelog:
>>>+ self.logger.removeHandler(self.consolelog)
>>>+ self.consolelog = None
>>>+ self.consolelog = logging.FileHandler(consolelogfile)
>>>+ bb.msg.addDefaultlogFilter(self.consolelog)
>>>+ format = bb.msg.BBLogFormatter("%(levelname)s:
>>>%(message)s")
>>>+ self.consolelog.setFormatter(format)
>>>+ self.logger.addHandler(self.consolelog)
>>>diff --git a/bitbake/lib/bb/ui/crumbs/hobeventhandler.py
>>>b/bitbake/lib/bb/ui/crumbs/hobeventhandler.py
>>>index 61af131..de228d9 100644
>>>--- a/bitbake/lib/bb/ui/crumbs/hobeventhandler.py
>>>+++ b/bitbake/lib/bb/ui/crumbs/hobeventhandler.py
>>>@@ -381,6 +381,9 @@ class HobHandler(gobject.GObject):
>>> def reset_build(self):
>>> self.build.reset()
>>>
>>>+ def get_logfile(self):
>>>+ return self.server.runCommand(["getVariable",
>>>+ "BB_CONSOLELOG"])
>>>+
>>> def get_parameters(self):
>>> # retrieve the parameters from bitbake
>>> params = {}
>>>diff --git a/bitbake/lib/bb/ui/crumbs/imagedetailspage.py
>>>b/bitbake/lib/bb/ui/crumbs/imagedetailspage.py
>>>index a5d0ad8..39ccfdc 100755
>>>--- a/bitbake/lib/bb/ui/crumbs/imagedetailspage.py
>>>+++ b/bitbake/lib/bb/ui/crumbs/imagedetailspage.py
>>>@@ -159,8 +159,10 @@ class ImageDetailsPage (HobPage):
>>> machine = self.builder.configuration.curr_mach
>>> base_image = self.builder.recipe_model.get_selected_image()
>>> layers = self.builder.configuration.layers
>>>+ log_file = self.builder.current_logfile
>>> pkg_num = "%s" %
>>>len(self.builder.package_model.get_selected_packages())
>>> else:
>>>+ log_file = None
>>> pkg_num = "N/A"
>>>
>>> self._remove_all_widget()
>>>@@ -204,9 +206,15 @@ class ImageDetailsPage (HobPage):
>>> image_table = HobViewTable(self.__columns__)
>>> image_table.set_model(self.image_store)
>>> image_table.connect("toggled", self.toggled_cb)
>>>+ view_buttons = gtk.VBox(False, 6)
>>> view_files_button = HobAltButton("View files")
>>> view_files_button.connect("clicked",
>>>self.view_files_clicked_cb,
>>>image_addr)
>>>-
>>>self.box_group_area.pack_start(self.DetailBox(widget=image_table,
>>>button=view_files_button), expand=True, fill=True)
>>>+ view_buttons.pack_start(view_files_button, expand=True,
>>>fill=True)
>>>+ if log_file:
>>>+ view_logs_button = HobAltButton("View logs")
>>>+ view_logs_button.connect("clicked",
>>>self.view_logs_clicked_cb, log_file)
>>>+ view_buttons.pack_end(view_logs_button, expand=True,
>>>fill=True)
>>>+
>>>self.box_group_area.pack_start(self.DetailBox(widget=image_table,
>>>button=view_buttons), expand=True, fill=True)
>>>
>>> # Machine, Base image and Layers
>>> layer_num_limit = 15
>>>@@ -258,6 +266,10 @@ class ImageDetailsPage (HobPage):
>>> def view_files_clicked_cb(self, button, image_addr):
>>> os.system("xdg-open /%s" % image_addr)
>>>
>>>+ def view_logs_clicked_cb(self, button, log_file):
>>>+ if log_file:
>>>+ os.system("xdg-open /%s" % log_file)
>>>+
>>> def refresh_package_detail_box(self, image_size):
>>> self.package_detail.update_line_widgets("Total image size: ",
>>>image_size)
>>>
>>>diff --git a/bitbake/lib/bb/ui/crumbs/runningbuild.py
>>>b/bitbake/lib/bb/ui/crumbs/runningbuild.py
>>>index 042902e..b42f9c8 100644
>>>--- a/bitbake/lib/bb/ui/crumbs/runningbuild.py
>>>+++ b/bitbake/lib/bb/ui/crumbs/runningbuild.py
>>>@@ -82,6 +82,9 @@ class RunningBuild (gobject.GObject):
>>> 'log-error' : (gobject.SIGNAL_RUN_LAST,
>>> gobject.TYPE_NONE,
>>> ()),
>>>+ 'log' : (gobject.SIGNAL_RUN_LAST,
>>>+ gobject.TYPE_NONE,
>>>+ (gobject.TYPE_STRING,
>>>gobject.TYPE_PYOBJECT,)),
>>> }
>>> pids_to_task = {}
>>> tasks_to_iter = {}
>>>@@ -119,6 +122,8 @@ class RunningBuild (gobject.GObject):
>>> parent = self.tasks_to_iter[(package, task)]
>>>
>>> if(isinstance(event, logging.LogRecord)):
>>>+ if event.taskpid == 0 or event.levelno > logging.INFO:
>>>+ self.emit("log", "handle", event)
>>> # FIXME: this is a hack! More info in Yocto #1433
>>> # http://bugzilla.pokylinux.org/show_bug.cgi?id=1433,
>>>temporarily
>>> # mask the error message as it's not informative for the
>>>user.
>>>@@ -204,6 +209,7 @@ class RunningBuild (gobject.GObject):
>>> self.tasks_to_iter[(package, task)] = i
>>>
>>> elif isinstance(event, bb.build.TaskBase):
>>>+ self.emit("log", "info", event._message)
>>> current = self.tasks_to_iter[(package, task)]
>>> parent = self.tasks_to_iter[(package, None)]
>>>
>>>@@ -284,6 +290,7 @@ class RunningBuild (gobject.GObject):
>>> pbar.set_text(event.msg)
>>>
>>> elif isinstance(event, bb.command.CommandFailed):
>>>+ self.emit("log", "error", "Command execution failed: %s"
>>>+ %
>>>(event.error))
>>> if event.error.startswith("Exited with"):
>>> # If the command fails with an exit code we're done,
>>>emit the
>>> # generic signal for the UI to notify the user @@
>>>-311,7 +318,41 @@ class RunningBuild (gobject.GObject):
>>> elif isinstance(event, bb.event.ParseCompleted) and pbar:
>>> pbar.hide()
>>> #using runqueue events as many as possible to update the
>>>progress bar
>>>+ elif isinstance(event, bb.event.MultipleProviders):
>>>+ self.emit("log", "info", "multiple providers are
>>>+ available
>>>for %s%s (%s)" \
>>>+ % (event._is_runtime and "runtime
>>>"
>>>or "", event._item, ", ".join(event._candidates)))
>>>+ self.emit("log", "info", "consider defining a
>>>PREFERRED_PROVIDER entry to match %s" % (event._item))
>>>+ elif isinstance(event, bb.event.NoProvider):
>>>+ if event._runtime:
>>>+ r = "R"
>>>+ else:
>>>+ r = ""
>>>+ if event._dependees:
>>>+ self.emit("log", "error", "Nothing %sPROVIDES '%s'
>>>+ (but
>>>%s %sDEPENDS on or otherwise requires it)" \
>>>+ % (r, event._item, ",
>>>".join(event._dependees), r))
>>>+ else:
>>>+ self.emit("log", "error", "Nothing %sPROVIDES '%s'" %
>>>(r, event._item))
>>>+ if event._reasons:
>>>+ for reason in event._reasons:
>>>+ self.emit("log", "error", "%s" % (reason))
>>>+ elif isinstance(event, bb.runqueue.runQueueTaskFailed):
>>>+ self.emit("log", "error", "Task %s (%s) failed with exit
>>>code '%s'" % (event.taskid, event.taskstring, event.exitcode))
>>>+ elif isinstance(event, bb.runqueue.sceneQueueTaskFailed):
>>>+ self.emit("log", "warn", "Setscene task %s (%s) failed
>>>+ with
>>>exit code '%s' - real task will be run instead" \
>>>+ % (event.taskid,
>>>+ event.taskstring,
>>>event.exitcode))
>>> elif isinstance(event, (bb.runqueue.runQueueTaskStarted,
>>>bb.runqueue.sceneQueueTaskStarted)):
>>>+ if isinstance(event, bb.runqueue.sceneQueueTaskStarted):
>>>+ self.emit("log", "info", "Running setscene task %d of
>>>+ %d
>>>(%s)" % \
>>>+ (event.stats.completed +
>>>event.stats.active + event.stats.failed + 1,
>>>+ event.stats.total,
>>>event.taskstring))
>>>+ else:
>>>+ if event.noexec:
>>>+ tasktype = 'noexec task'
>>>+ else:
>>>+ tasktype = 'task'
>>>+ self.emit("log", "info", "Running %s %s of %s (ID:
>>>+ %s,
>>>%s)" % \
>>>+ (tasktype,
>>>event.stats.completed + event.stats.active + event.stats.failed + 1,
>>>+ event.stats.total,
>>>event.taskid, event.taskstring))
>>> message = {}
>>> message["eventname"] = bb.event.getName(event)
>>> num_of_completed = event.stats.completed +
>>>event.stats.failed @@ -320,6 +361,18 @@ class RunningBuild
>>>(gobject.GObject):
>>> message["title"] = ""
>>> message["task"] = event.taskstring
>>> self.emit("task-started", message)
>>>+ else:
>>>+ if not isinstance(event, (bb.event.BuildBase,
>>>+ bb.event.StampUpdate,
>>>+ bb.event.ConfigParsed,
>>>+ bb.event.RecipeParsed,
>>>+ bb.event.RecipePreFinalise,
>>>+ bb.runqueue.runQueueEvent,
>>>+ bb.runqueue.runQueueExitWait,
>>>+ bb.event.OperationStarted,
>>>+ bb.event.OperationCompleted,
>>>+ bb.event.OperationProgress)):
>>>+ self.emit("log", "error", "Unknown event: %s" %
>>>+ (error))
>>>
>>> return
>>>
>>>--
>>>1.7.6
>>>
>>>
>>>_______________________________________________
>>>bitbake-devel mailing list
>>>bitbake-devel at lists.openembedded.org
>>>http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/bitbake-devel
>>
>
---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47
This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
More information about the bitbake-devel
mailing list