[bitbake-devel] [PATCH] bitbake: server/xmlrpc/prserv: Increase timeout to default xmlrpc server

[Richard Purdie]

On Wed, 2013-08-28 at 08:59 -0500, Peter A. Bigot wrote:
> On 08/28/2013 08:39 AM, Jason Wessel wrote:
> > On 08/28/2013 08:04 AM, Peter A. Bigot wrote:
> >> On a heavily-loaded host with local PR server the default 5 second timeout
> >> produces too-frequent errors:
> >>
> >>    ERROR: Can NOT get PRAUTO, exception timed out
> >>    ERROR: Function failed: package_get_auto_pr
> >>
> >> Since this error aborts the build a generous timeout seems appropriate.
> >>
> >> Signed-off-by: Peter A. Bigot <pab at pabigot.com>
> >> ---
> >>   lib/bb/server/xmlrpc.py | 2 +-
> >>   1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/lib/bb/server/xmlrpc.py b/lib/bb/server/xmlrpc.py
> >> index 4dee5d9..bb87fd7 100644
> >> --- a/lib/bb/server/xmlrpc.py
> >> +++ b/lib/bb/server/xmlrpc.py
> >> @@ -78,7 +78,7 @@ class BBTransport(xmlrpclib.Transport):
> >>               h.putheader("Bitbake-token", self.connection_token)
> >>           xmlrpclib.Transport.send_content(self, h, body)
> >>   
> >> -def _create_server(host, port, timeout = 5):
> >> +def _create_server(host, port, timeout = 20):
> >>       t = BBTransport(timeout)
> >>       s = xmlrpclib.Server("http://%s:%d/" % (host, port), transport=t, allow_none=True)
> >>       return s, t
> >
> > I would go so far as to make this 60 seconds and or have it a configurable parameter.
> >
> > Previously the timeout was infinite.   I have observed process creation lagging by 30-45 seconds on a server with a load average of +300.   The new bitbake python code with the reduced timeout is not yet running on our edge case testing environment, but I do expect to hit the same issue.
> 
> Not sure when the timeout was added, but I believe it was before the 
> modifications in the last few days that moved it to this function; I've 
> been having this problem since switching to poky master.
> 
> 60 seconds would be fine with me; I could update the patch for that.  A 
> configurable parameter would be better but it wasn't obvious how to do 
> it, so if people prefer that approach I'd rather a bitbake maintainer 
> take over from here.

The downside is that if something goes wrong this ends up leaving
bitbake hanging for 60 seconds at exit whilst it tries to connect to a
server which is never going to exist. I'm rather frustrated that the PR
service is so slow since this will block the packaging process for that
length or time.

With that in mind I've radically improved the performance of the server
with threading. Can people retest with master and see how things behave
now?

Cheers,

Richard