[OE-core] [PATCH 0/2] Restore performance of rpm in Docker containers
Peter Kjellerstedt
peter.kjellerstedt at axis.com
Thu May 10 22:20:40 UTC 2018
As recently discussed on the Yocto list in the "ROOTFS_RPM_DEBUG
undocumented" thread, there are performance issues with dnf/rpm
when building in a Docker container. The problem is due to the
Docker container reporting the maximum number of open files as
unlimited and there is code in rpm that loops over all potentially
open file descriptors. Alexander Kanavin supplied a patch for rpm
(see commit 6f1822e5 in meta) that was supposed to correct this, but
unfortunately it only solved a part of the problem.
To really solve the problem, I reverted Alexander's patch, and
applied a new one that solves both the problematic code identified
by Alexander, and an identical code path in another function.
It is possible to reproduce the problem without Docker by increasing
the maximum number of open files. This can be done by editing
/etc/security/limits.conf and adding:
* - nofile 100000000
where 100000000 is the maximum number of open files. It is also
very likely that you need to increase the global limit by running:
sudo sysctl -w fs.nr_open=100000000
After that, ssh to localhost to get a new session where the new
settings are active. Then you can use `ulimit -n <number>` to set
the max you want to try.
I also did som measurements when building core-image-minimal. Here
are the times I got for the do_rootfs task with different maximums:
Max files Time
--------- -----
1024 30 seconds
1048576 50 seconds
10000000 3 minutes 40 seconds
100000000 31 minutes
//Peter
The following changes since commit f4c938c47424424b89cde2269bd92cebc9a6ac1a:
packagegroup: Do not add libssp to SDK (2018-05-09 10:47:51 +0100)
are available in the git repository at:
git://push.yoctoproject.org/poky-contrib pkj/rpm-performance
Peter Kjellerstedt (2):
Revert "rpm: add a patch to help with Docker performance issues"
rpm: Assume a max limit of 1024 open files
...001-Assume-a-max-limit-of-1024-open-files.patch | 61 ++++++++++++++++++++++
...FD_CLOEXEC-on-opened-files-before-exec-fr.patch | 49 -----------------
meta/recipes-devtools/rpm/rpm_4.14.1.bb | 3 +-
3 files changed, 63 insertions(+), 50 deletions(-)
create mode 100644 meta/recipes-devtools/rpm/files/0001-Assume-a-max-limit-of-1024-open-files.patch
delete mode 100644 meta/recipes-devtools/rpm/files/0001-Revert-Set-FD_CLOEXEC-on-opened-files-before-exec-fr.patch
--
2.12.0
More information about the Openembedded-core
mailing list