[OE-core] [PATCH] local.conf.sample.extended: Document how to limit parallel fetch tasks

[Richard Purdie]

On Wed, 2020-02-19 at 14:05 -0800, Andre McCurdy wrote:
> On Wed, Feb 19, 2020 at 1:43 PM Richard Purdie
> <richard.purdie at linuxfoundation.org> wrote:
> > [YOCTO #13235]
> > 
> > Signed-off-by: Richard Purdie <richard.purdie at linuxfoundation.org>
> > ---
> >  meta/conf/local.conf.sample.extended | 8 ++++++++
> >  1 file changed, 8 insertions(+)
> > 
> > diff --git a/meta/conf/local.conf.sample.extended
> > b/meta/conf/local.conf.sample.extended
> > index fcf4a9b7c1a..443b4e83f9a 100644
> > --- a/meta/conf/local.conf.sample.extended
> > +++ b/meta/conf/local.conf.sample.extended
> > @@ -23,6 +23,14 @@
> >  #
> >  # For a quad-core machine, BB_NUMBER_THREADS = "4", PARALLEL_MAKE
> > = "-j 4" would
> >  # be appropriate for example.
> > +#
> > +# Some users are behind firewalls or use servers where the number
> > of parallel connections
> > +# is limited. In such cases you can limit the number of fetch
> > tasks which run in parallel by
> > +# setting the option below, in this case limiting to a maximum of
> > 4 fetch tasks in parallel:
> > +#
> > +#do_fetch[number_threads] = "4"
> 
> How does this interact with BB_NUMBER_THREADS? If, for example, both
> are set to 4 does it mean there could be 4 fetch threads and 4
> compile threads running in parallel?

BB_NUMBER_THREADS is an upper limit which includes fetch tasks.

> A related issue when the number of parallel connections is limited is
> parsing a large number of AUTOREV recipes. From what I remember the
> workaround for that was to set BB_NUMBER_PARSE_THREADS. Maybe that
> should be documented in local.conf.sample.extended too? Perhaps even
> better would be to define a single new variable which limits the
> number of parallel connections in both cases?

I don't know how you'd implement a total number of connections limit
from a code perspective :/.

You'd probably need something like a make job server where you obtained
tokens before making connections and I can see this becoming "fun"
since you'd have to markup every code section where a network
connection was made.

We have a conflict between code complexity, maintainability,
performance and functionality and I'm not sure the use case here
warrants the complexity...

Cheers,

Richard