[OE-core] [PATCH 1/2] package_rpm.bbclass: use multithreaded xz compression
Alexander Kanavin
alexander.kanavin at linux.intel.com
Fri Jun 2 11:40:55 UTC 2017
On 06/01/2017 10:02 PM, Andre McCurdy wrote:
>> RPM's default is single-threaded gz; the change greatly helps with
>> both buildtimes (when there is a small number of large-sized packages)
>> and disk space taken by resulting rpms.
>
> Do you have any test results for that? Obviously build times will be
> longer when building with a single CPU since xz compression needs more
> CPU than gzip, but I'm curious how many CPUs are required before the
> multiple threads win over and build times actually get shorter. 4? 8?
Yes; it's in the cover letter.
> One other concern with parallel xz compression is memory usage. Single
> threaded xz needs more memory than gzip and running multiple xz
> compression threads together will push the memory usage up again.
> Maybe not an issue if the peak is still acceptable, but something to
> be aware of (I've given up trying to build in 4GB, but hopefully 8GB
> build machines still have some life left in them...).
I haven't seen anything resembling low-memory or OOM in my testing, even
when running 44 rpm instances at the same time, each with their own
several threads, each thread handling a package, and compression for the
package again being done in a multi-threaded way. But then I have 22
cores and 32G RAM here.
So do try the patches please, and report the findings, YMMV.
Alex
More information about the Openembedded-core
mailing list