[OE-core] World builds HW Was: [PATCH 04/12] gnome-icon-theme: remove the recipe

Wed Oct 14 13:01:00 UTC 2015

On Tue, Oct 13, 2015 at 04:43:44PM -0400, Randy MacLeod wrote:
> On 2015-09-18 01:56 PM, Tom King wrote:
> > Martin, Randy et.al <http://et.al>.:
> >
> > I'm moving around things such that I can give Martin a full builder:
> >
> > 24Cores, 256GB RAM,  up to 3TB storage from the donation that HP made to
> > WebOS-Ports (as I consider it essential to webOS-Ports that OE has a
> > strong foundation)
> >
> > Tom
> 
> Thanks Tom.
> I'm back from vacation, cleaning up old emails.
> 
> Did the upgrade help Martin?

I'm still waiting for access to that server.

> Do you need additional systems?

More builders are always welcome. In ideal case I would trigger all 3
architectures in parallel on different builders.

> ../Randy
> >
> > On Fri, Sep 18, 2015 at 10:30 AM, Martin Jansa <martin.jansa at gmail.com
> > <mailto:martin.jansa at gmail.com>> wrote:
> >
> >     On Fri, Sep 18, 2015 at 12:24:00PM -0400, Randy MacLeod wrote:
> >      > On 2015-09-07 11:14 AM, Martin Jansa wrote:
> >      > > Full jenkins build test takes over 3 days (depending on sstate
> >     reuse,
> >      > > one MACHINE can take more than 24 hours), so if I add them to
> >      > > master-next after it's merged to oe-core it will take 3-6 days to
> >      > > build-test them (6 days if they are sent just after current
> >     queue is
> >      > > sent for build).
> >      >
> >      > Hi Martin,
> >
> >     Hi Randy,
> >
> >      > Replying privately but you can add the list back in your
> >      > reply if you like.
> >
> >     OK, adding both MLs
> >
> >      > I never build all of meta-oe but a day seems like a long time.
> >      > On a 3-4 year old 24 core system, with 64 GB RAM and
> >      > 2 large disks, I build oe-core + a whitelist of meta-oe + misc
> >     layers in
> >      > 5 hours using 18.3 of the 24 cores on average.
> >      > What are the specs of your build machine?
> >
> >     It's one of VMs running on box donated by HP to webos-ports project. It
> >     has access to 8 E5-2630L 0 @ 2.00GHz cores (from IIRC 24) and slow IO
> >     which we try to compensate a bit by using tmpfs for TMPDIR, VM has 80G
> >     RAM, so I'm using 72G for tmpfs.
> >
> >     Tom: please fill-in the gaps, I don't know any details about the real HW
> >     it's running on.
> >
> >     The same machine is also running some webos-ports builds sometimes.
> >
> >     BUT be aware that my world build is much bigger than what you're
> >     probably building and changes in oe-core almost always ensure very low
> >     sstate reuse.
> >
> >     e.g. last build with only small oe-core upgrade since previous build:
> >     NOTE: Tasks Summary: Attempted 33283 tasks of which 5252 didn't need
> >     to be rerun and 33 failed.
> >     NOTE: Build completion summary:
> >     NOTE:   do_populate_sysroot: 21% sstate reuse (485 setscene, 1750
> >     scratch)
> >     NOTE:   do_package_qa: 12% sstate reuse (262 setscene, 1760 scratch)
> >     NOTE:   do_package: 4% sstate reuse (76 setscene, 1750 scratch)
> >     NOTE:   do_packagedata: 13% sstate reuse (281 setscene, 1750 scratch)
> >     NOTE:   do_package_write_ipk: 12% sstate reuse (262 setscene, 1756
> >     scratch)
> >     NOTE:   do_populate_lic: 22% sstate reuse (521 setscene, 1806 scratch)
> >
> >     real    1485m40.070s
> >     user    4166m49.125s
> >     sys     2825m50.068s
> >
> >     BTW: you can see all this information in the world logs I'm sending
> >     to MLs
> >     (but not all the builds make it to ML report).
> >
> >      > A decent build box these days has 32+ cores, 64+ GB RAM  and
> >      > and SSD or RAID disk filesystem. The costs of such a system
> >      > are significant but worth it given the number of people's time
> >      > involved in meta-oe.
> >
> >     That's true, but I have to do with what I have available :).
> >
> >      > Also, it would be nice if OE had a few such machines each
> >      > working on builds for a MACHINE to reduce your turn-around
> >      > to more like a day or so. :)
> >
> >     Yes, it would be nice, I'm trying to get Tom King to set another VM like
> >     this in another location (another HP donated server), hopefully he will
> >     finish it soon.
> >
> >     It's all hosted together with OE infrastructure, bandwidth is
> >     limited, but
> >     all I need for these builds is to fetch sources and upload the logs,
> >     there
> >     are no package feeds or sstate from these builds usable outside, but
> >     now it
> >     rsync the downloaded sources to sources.openembedded.org
> >     <http://sources.openembedded.org>
> >
> >     If you know about some idling build servers, I would like to give them
> >     something to do :).
> >
> >     Regards,
> >
> >     --
> >     Martin 'JaMa' Jansa     jabber: Martin.Jansa at gmail.com
> >     <mailto:Martin.Jansa at gmail.com>
> >
> >
> 
> 
> -- 
> # Randy MacLeod. SMTS, Linux, Wind River
> Direct: 613.963.1350 | 350 Terry Fox Drive, Suite 200, Ottawa, ON, 
> Canada, K2K 2W5

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa at gmail.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20151014/7773290a/attachment-0002.sig>