[OE-core] [PATCH] glibc-locale: Rewrite do_install using install utility instead of cp
Peter Kjellerstedt
peter.kjellerstedt at axis.com
Sun Feb 10 06:25:10 UTC 2019
> -----Original Message-----
> From: openembedded-core-bounces at lists.openembedded.org <openembedded-
> core-bounces at lists.openembedded.org> On Behalf Of Peter Kjellerstedt
> Sent: den 10 februari 2019 02:29
> To: Richard Purdie <richard.purdie at linuxfoundation.org>; Khem Raj
> <raj.khem at gmail.com>; openembedded-core at lists.openembedded.org
> Subject: Re: [OE-core] [PATCH] glibc-locale: Rewrite do_install using
> install utility instead of cp
>
> > -----Original Message-----
> > From: openembedded-core-bounces at lists.openembedded.org <openembedded-
> > core-bounces at lists.openembedded.org> On Behalf Of Richard Purdie
> > Sent: den 7 februari 2019 10:01
> > To: Khem Raj <raj.khem at gmail.com>; openembedded-
> > core at lists.openembedded.org
> > Subject: Re: [OE-core] [PATCH] glibc-locale: Rewrite do_install using
> > install utility instead of cp
> >
> > On Wed, 2019-02-06 at 16:35 -0800, Khem Raj wrote:
> > > This has been a constant source of trouble for build failures due
> to
> > host-user-contaminated QA
> > > errors of sort
> > >
> > > ERROR: QA Issue: glibc-locale: /glibc-binary-localedata-ca-
> > es+valencia/usr/lib/locale/ca_ES at valencia/LC_MONETARY is owned by uid
> > 3004, which is the same as the user running bitbake. This may be due
> to
> > host contamination [host-user-contaminated]
> > >
> > > So far we have tried to mould cp command into not carrying the
> build
> > > user permissions into install area but it is never entirely fixed
> > since
> > > the issue keeps popping up in various scenes
> > >
> > > This patch replaces use of cp with install utility and specifies
> > install
> > > mode for files explcitly
> > >
> > > Signed-off-by: Khem Raj <raj.khem at gmail.com>
> > > ---
> > > meta/recipes-core/glibc/glibc-locale.inc | 44 ++++++++++++++------
> --
> > --
> > > 1 file changed, 25 insertions(+), 19 deletions(-)
> > >
> > > diff --git a/meta/recipes-core/glibc/glibc-locale.inc
> b/meta/recipes-
> > core/glibc/glibc-locale.inc
> > > index 6384f9cbf1..9b256a5108 100644
> > > --- a/meta/recipes-core/glibc/glibc-locale.inc
> > > +++ b/meta/recipes-core/glibc/glibc-locale.inc
> > > @@ -72,27 +72,33 @@ FILES_localedef = "${bindir}/localedef"
> > > LOCALETREESRC = "${COMPONENTS_DIR}/${PACKAGE_ARCH}/glibc-stash-
> > locale"
> > >
> > > do_install () {
> > > - mkdir -p ${D}${bindir} ${D}${datadir}
> > > - if [ -n "$(ls ${LOCALETREESRC}/${bindir})" ]; then
> > > - cp -R --no-dereference --preserve=mode,links
> > ${LOCALETREESRC}/${bindir}/* ${D}${bindir}
> > > - fi
> > > - if [ -n "$(ls ${LOCALETREESRC}/${localedir})" ]; then
> > > - mkdir -p ${D}${localedir}
> > > - cp -R --no-dereference --preserve=mode,links
> > ${LOCALETREESRC}/${localedir}/* ${D}${localedir}
> > > - fi
> > > + install -d ${D}${bindir}
> > > + find "${LOCALETREESRC}/${bindir}" -maxdepth 1 -type f \
> > > + -exec install -m 0755 -t "${D}${bindir}" {} \;
> > > +
> > > + for d in . $(find "${LOCALETREESRC}/${localedir}" -type d
> -
> > printf '%P ') ; do
> > > + install -d "${D}${localedir}/$d"
> > > + find "${LOCALETREESRC}/${localedir}/$d" -maxdepth
> 1
> > -type f \
> > > + -exec install -m 0644 -t "${D}${localedir}/$d" {}
> \;
> > > + done
> > > if [ ${@d.getVar('PACKAGE_NO_GCONV')} -eq 0 ]; then
> > > - mkdir -p ${D}${libdir}
> > > - if [ -e ${LOCALETREESRC}/${libdir}/gconv ]; then
> > > - cp -R --no-dereference --
> > preserve=mode,links ${LOCALETREESRC}/${libdir}/gconv ${D}${libdir}
> > > - fi
> > > - if [ -e ${LOCALETREESRC}/${datadir}/i18n ]; then
> > > - cp -R --no-dereference --
> > preserve=mode,links ${LOCALETREESRC}/${datadir}/i18n ${D}${datadir}
> > > - fi
> > > - fi
> > > - if [ -e ${LOCALETREESRC}/${datadir}/locale ]; then
> > > - cp -R --no-dereference --preserve=mode,links
> > ${LOCALETREESRC}/${datadir}/locale ${D}${datadir}
> > > + for d in . $(find
> "${LOCALETREESRC}/${libdir}/gconv"
> > -type d -printf '%P ') ; do
> > > + install -d "${D}${libdir}/gconv/$d"
> > > + find "${LOCALETREESRC}/${libdir}/gconv/$d"
> -
> > maxdepth 1 -type f \
> > > + -exec install -m 0755 -t
> > "${D}${libdir}/gconv/$d" {} \;
> > > + done
> > > + for d in . $(find
> "${LOCALETREESRC}/${datadir}/i18n"
> > -type d -printf '%P ') ; do
> > > + install -d "${D}${datadir}/i18n/$d"
> > > + find "${LOCALETREESRC}/${datadir}/i18n/$d"
> -
> > maxdepth 1 -type f \
> > > + -exec install -m 0644 -t
> > "${D}${datadir}/i18n/$d" {} \;
> > > + done
> > > fi
> > > - cp -R --no-dereference --preserve=mode,links
> > ${LOCALETREESRC}/SUPPORTED ${WORKDIR}
> > > + for d in . $(find "${LOCALETREESRC}/${datadir}/locale" -
> type
> > d -printf '%P ') ; do
> > > + install -d "${D}${datadir}/locale/$d"
> > > + find "${LOCALETREESRC}/${datadir}/locale/$d" -
> > maxdepth 1 -type f \
> > > + -exec install -m 0644 -t
> "${D}${datadir}/locale/$d"
> > {} \;
> > > + done
> > > + install -m 0644 ${LOCALETREESRC}/SUPPORTED
> > ${WORKDIR}/SUPPORTED
> > > }
> > >
> > > inherit libc-package
> >
> > The trouble is this is a workaround. The cp commands should work and
> > there is some underlying issue in pseudo causing this. We really need
> > to figure out what that is since this will likely just mean we see
> the
> > problem somewhere else :(
> >
> > I still suspect some kind of inode number reuse problem which these
> cp
> > commands trigger...
> >
> > Cheers,
> >
> > Richard
>
> For the record, even if I think the patch was the right thing to do,
> it did not help the situation. The first build I did after it was
> integrated, I was bit by the following:
>
> Failed to determine the user name of UID 323 for
> tmp/work/core2-64-poky-linux/glibc-locale/2.29-r0/package/usr/lib/locale
>
> where UID 323 is my UID. If you do not recognize the error message,
> it is because it comes from a local patch we have for rpm to protect
> us from any incorrect UIDs/GIDs causing incorrect RPMs to be generated
> (see below). I checked the build directory and the /usr/lib/locale
> directory was the only file or directory with an incorrect owner,
> which is typical of these pseudo related problems with incorrect
> owners.
>
> [rpm will by default silently fall back to use root for any files
> where it cannot determine the name of the user or group based on
> its UID/GID. This can be due to missing information in the
> /etc/passwd or /etc/groups files, typically due to relying on
> another recipe in DEPENDS to create the user without also having
> an RDEPENDS on the package that creates the user, or because of
> pseudo messing up. After this bit us once where a device in /dev
> ended up in an rpm in the sstate cache with root as group instead
> of the intended video group, which in turn caused the video
> application to fail as it was no longer allowed to open its device,
> I created the patch. It is causing a fair bit of grief as it causes
> builds to fail randomly, but at least we don't end up with an
> incorrect sstate anymore, which is worse. I am not sure this patch
> is suitable for integration to OE-Core, but I have attached it if
> anyone is interested.]
>
> //Peter
Ok, this just got a lot more weird. At the time I wrote the mail
above, I had only tried to rebuild once. Now I have done it a number
of more times, and the result is not random any more. Every build
fails the same way. I've done `bitbake glibc-locale -c cleansstate`
followed by `bitbake glibc-locale` and the result is consistent.
What I have noted now, is that it is _not_ do_install that causes
the problem. The /usr/lib/locale directory created in ${D} has
the correct owner. It is when the data is copied to ${PKGD} that
it somehow ends up with the wrong owner. I cannot tell from a quick
glance what differs /usr/lib/locale from, e.g., /usr/lib/gconv,
where the latter does get the correct owner.
Unfortunately, I do not have more time to look at this right now,
but I thought I should share this information with you. If anyone
has any ideas of what to look at to determine why
${PKGD}/usr/lib/locale ends up with the wrong owner, I'm all ears.
It would also be interesting to hear if the result is the same for
anyone else with the patch I provided in my previous mail applied.
//Peter
More information about the Openembedded-core
mailing list