[OE-core] [PATCH 2/3] rm_work_and_downloads.bbclass: more aggressively minimize disk usage

[Randy Witt]

Hi Patrick,

On Fri, Jan 6, 2017 at 1:55 AM, Patrick Ohly <patrick.ohly at intel.com> wrote:

> rm_work.bbclass never deletes downloaded files, even if they are not
> going to be needed again during the
> build. rm_work_and_downloads.bbclass is more aggressive in minimizing
> the used disk space during a build, but has other disadvantages:
> - sources required by different recipes need to be fetched once per
>   recipe, not once per build
> - incremental builds do not work reliably because sources get
>   removed without ensuring that sources gets fetched again
>
> That makes rm_work_and_downloads.bbclass useful for one-time builds in
> a constrained environment (like a CI system), but not for general use.
>
>
I'd rather see this be a new class independent from rm_work. People can
always in inherit both rm_work and rm_download.

That being said, since this doesn't operate at the fetch level, i.e. only
remove things that were fetched as part of the current invocation of
bitbake, I don't see why the user couldn't just "rm -rf downloads"
themselves as part of the ci teardown.


> Signed-off-by: Patrick Ohly <patrick.ohly at intel.com>
> ---
>  meta/classes/rm_work_and_downloads.bbclass | 33
> ++++++++++++++++++++++++++++++
>  1 file changed, 33 insertions(+)
>  create mode 100644 meta/classes/rm_work_and_downloads.bbclass
>
> diff --git a/meta/classes/rm_work_and_downloads.bbclass
> b/meta/classes/rm_work_and_downloads.bbclass
> new file mode 100644
> index 0000000..7c00bea
> --- /dev/null
> +++ b/meta/classes/rm_work_and_downloads.bbclass
> @@ -0,0 +1,33 @@
> +# Author:       Patrick Ohly <patrick.ohly at intel.com>
> +# Copyright:    Copyright (C) 2015 Intel Corporation
> +#
> +# This file is licensed under the MIT license, see COPYING.MIT in
> +# this source distribution for the terms.
> +
> +# This class is used like rm_work:
> +# INHERIT += "rm_work_and_downloads"
> +#
> +# In addition to removing local build directories of a recipe, it also
> +# removes the downloaded source. This is achieved by making the DL_DIR
> +# recipe-specific. While reducing disk usage, it increases network usage
> (for
> +# example, compiling the same source for target and host implies
> downloading
> +# the source twice).
> +#
> +# Because the "do_fetch" task does not get re-run after removing the
> downloaded
> +# sources, this class is also not suitable for incremental builds.
> +#
> +# Where it works well is in well-connected build environments with limited
> +# disk space (like TravisCI).
> +
> +inherit rm_work
> +
> +# This would ensure that the existing do_rm_work() removes the downloads,
> +# but does not work because some recipes have a circular dependency
> between
> +# WORKDIR and DL_DIR (via ${SRCPV}?).
> +# DL_DIR = "${WORKDIR}/downloads"
> +
> +# Instead go up one level and remove ourself.
> +DL_DIR = "${BASE_WORKDIR}/${MULTIMACH_TARGET_SYS}/${PN}/downloads"
> +do_rm_work_append () {
> +    rm -rf ${DL_DIR}
> +}
> --
> 2.1.4
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core at lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20170106/a9471efe/attachment-0002.html>