[OE-core] [PATCH 0/1] Add new mirror archiver mode

[Paul Barker]

On Thu, 26 Sep 2019, at 21:31, Richard Purdie wrote:
> On Tue, 2019-09-24 at 10:06 +0100, Paul Barker wrote:
> > This patch allows the creation of a full source mirror using the
> > archiver
> > bbclass. Using the archiver allows us to make use of copyleft license
> > filtering, recipe type filtering and other features that are not
> > available if
> > we just copy the contents of the downloads directory itself to create
> > a
> > mirror. We also get the advantage of sstate caching and we don't have
> > to
> > throw away the entire downloads directory before building to keep
> > sources for
> > other builds out of our mirror.
> > 
> > The resulting mirror may be published and used as an entry in
> > PREMIRRORS or
> > MIRRORS for a subsequent build. The functionality is explained in
> > more detail
> > in the patch commit message and the additions to the documentation
> > comments.
> > 
> > We've been using this patch in our Oryx Linux distro for a few months
> > now to
> > confirm that it works as expected. All feedback is welcome,
> > especially from
> > people who have worked on the archiver bbclass before.
> > 
> > Mirror creation was tested with the following additions to
> > local.conf:
> > 
> >     INHERIT += "archiver"
> >     BB_GENERATE_MIRROR_TARBALLS = "1"
> >     ARCHIVER_MODE[src] = "mirror"
> >     ARCHIVER_MODE[mirror] = "combined"
> >     ARCHIVER_MIRROR_EXCLUDE = "file://"
> >     COPYLEFT_LICENSE_INCLUDE = "*"
> > 
> > The resulting mirror directory was moved out of the tmpdir and a
> > rebuild was
> > tested with the following additions to local.conf:
> > 
> >     BB_NO_NETWORK = "1"
> >     INHERIT += "own-mirrors"
> >     SOURCE_MIRROR_URL = "file://${TOPDIR}/mirror"
> > 
> > Paul Barker (1):
> >   archiver.bbclass: Add new mirror archiver mode
> > 
> >  meta/classes/archiver.bbclass | 136 +++++++++++++++++++++++++++++---
> > --
> >  1 file changed, 117 insertions(+), 19 deletions(-)
> 
> This is coming in a bit late to be in 3.0 as we're way past feature
> freeze. Have you given thought to regression testing? I'd also really
> like to see some tests. There are some already:
> 
> meta/lib/oeqa/selftest/cases/archiver.py 
> 
> and whilst I'd not going to claim they're wonderful, it would be good
> to extend as we add new functionality.

I've parked this for now, as you say it's a bit late for 3.0. I'll resubmit with some extra test cases early in the 3.1 cycle.

-- 
Paul Barker