[OE-core] Connectivity check uris

Richard Purdie richard.purdie at linuxfoundation.org
Thu Jan 23 13:26:23 UTC 2020


On Thu, 2020-01-23 at 08:14 -0500, Jean-Marie LEMETAYER wrote:
> Hi folks,
> 
> I have noticed some hang-ups at the beginning of my builds on the
> master branch.
> I have search a little and discovered that the connectivity check
> made by poky
> was the root cause. In fact connectivity check use the
> CONNECTIVITY_CHECK_URIS
> variable which is by default set to "https://www.example.com/".
> 
> So I have made some tests with a little script which gets the URI 100
> times:
> 
> 	$ cat connectivity_check.sh
> 	#!/bin/sh
> 	connectivity_check()
> 	{
> 		local ok=0
> 		for i in $(seq 100)
> 		do
> 			$@ > /dev/null && ok=$(($ok + 1))
> 		done
> 		echo $ok $@
> 	}
> 	connectivity_check wget -qO- --timeout=1 --tries=1 $1
> 	connectivity_check curl -s --connect-timeout 1 $1
> 
> Note that in the script, I use wget and curl to see if there are any
> differences
> but I don't think that's a thing.
> 
> Here are the results:
> 	$ time ./connectivity_check.sh https://example.com
> 	45 wget -qO- --timeout=1 --tries=1 https://example.com
> 	45 curl -s --connect-timeout 1 https://example.com
> 
> 	real	2m23.595s
> 	user	0m3.352s
> 	sys	0m1.834s
> 
> 	$ time ./connectivity_check.sh https://openembedded.org
> 	100 wget -qO- --timeout=1 --tries=1 https://openembedded.org
> 	100 curl -s --connect-timeout 1 https://openembedded.org
> 
> 	real	4m36.188s
> 	user	0m3.853s
> 	sys	0m1.761s
> 
> 	$ time ./connectivity_check.sh https://google.com
> 	100 wget -qO- --timeout=1 --tries=1 https://google.com
> 	100 curl -s --connect-timeout 1 https://google.com
> 
> 	real	0m36.328s
> 	user	0m4.265s
> 	sys	0m2.028s
> 
> So "example.com" is clearly not a good domain to do connectivity
> checks.
> 
> The "openembedded.org" domain is good but have a slow response time.
> 
> And finally "google.com" which have all sort of speedy networking
> stuff is very
> efficient.
> 
> ---
> 
> I think the CONNECTIVITY_CHECK_URIS variable must be updated and not
> use the
> "example.com" domain anymore. Using "google.com" may be the best
> option. But
> eetting up a custom domain like "connectivitycheck.openembedded.org"
> could be
> another idea.
> 
> Any ideas 

I think if you look at the history we did use google.com however that
caused problems for people in geos where that is blocked (e.g. PRC).

We may want to move the default back given the performance isse, not
sure...

Cheers,

Richard



More information about the Openembedded-core mailing list