[OE-core] [PATCH] glibc-locale: Rewrite do_install using install utility instead of cp

Sun Feb 10 01:29:21 UTC 2019

> -----Original Message-----
> From: openembedded-core-bounces at lists.openembedded.org <openembedded-
> core-bounces at lists.openembedded.org> On Behalf Of Richard Purdie
> Sent: den 7 februari 2019 10:01
> To: Khem Raj <raj.khem at gmail.com>; openembedded-
> core at lists.openembedded.org
> Subject: Re: [OE-core] [PATCH] glibc-locale: Rewrite do_install using
> install utility instead of cp
> 
> On Wed, 2019-02-06 at 16:35 -0800, Khem Raj wrote:
> > This has been a constant source of trouble for build failures due to
> host-user-contaminated QA
> > errors of sort
> >
> > ERROR: QA Issue: glibc-locale: /glibc-binary-localedata-ca-
> es+valencia/usr/lib/locale/ca_ES at valencia/LC_MONETARY is owned by uid
> 3004, which is the same as the user running bitbake. This may be due to
> host contamination [host-user-contaminated]
> >
> > So far we have tried to mould cp command into not carrying the build
> > user permissions into install area but it is never entirely fixed
> since
> > the issue keeps popping up in various scenes
> >
> > This patch replaces use of cp with install utility and specifies
> install
> > mode for files explcitly
> >
> > Signed-off-by: Khem Raj <raj.khem at gmail.com>
> > ---
> >  meta/recipes-core/glibc/glibc-locale.inc | 44 ++++++++++++++--------
> --
> >  1 file changed, 25 insertions(+), 19 deletions(-)
> >
> > diff --git a/meta/recipes-core/glibc/glibc-locale.inc b/meta/recipes-
> core/glibc/glibc-locale.inc
> > index 6384f9cbf1..9b256a5108 100644
> > --- a/meta/recipes-core/glibc/glibc-locale.inc
> > +++ b/meta/recipes-core/glibc/glibc-locale.inc
> > @@ -72,27 +72,33 @@ FILES_localedef = "${bindir}/localedef"
> >  LOCALETREESRC = "${COMPONENTS_DIR}/${PACKAGE_ARCH}/glibc-stash-
> locale"
> >
> >  do_install () {
> > -	mkdir -p ${D}${bindir} ${D}${datadir}
> > -	if [ -n "$(ls ${LOCALETREESRC}/${bindir})" ]; then
> > -		cp -R --no-dereference --preserve=mode,links
> ${LOCALETREESRC}/${bindir}/* ${D}${bindir}
> > -	fi
> > -	if [ -n "$(ls ${LOCALETREESRC}/${localedir})" ]; then
> > -		mkdir -p ${D}${localedir}
> > -		cp -R --no-dereference --preserve=mode,links
> ${LOCALETREESRC}/${localedir}/* ${D}${localedir}
> > -	fi
> > +        install -d ${D}${bindir}
> > +        find "${LOCALETREESRC}/${bindir}" -maxdepth 1 -type f \
> > +        -exec install -m 0755 -t "${D}${bindir}" {} \;
> > +
> > +        for d in . $(find "${LOCALETREESRC}/${localedir}" -type d -
> printf '%P ') ; do
> > +                install -d "${D}${localedir}/$d"
> > +                find "${LOCALETREESRC}/${localedir}/$d" -maxdepth 1
> -type f \
> > +                -exec install -m 0644 -t "${D}${localedir}/$d" {} \;
> > +        done
> >  	if [ ${@d.getVar('PACKAGE_NO_GCONV')} -eq 0 ]; then
> > -		mkdir -p ${D}${libdir}
> > -		if [ -e ${LOCALETREESRC}/${libdir}/gconv ]; then
> > -			cp -R --no-dereference --
> preserve=mode,links ${LOCALETREESRC}/${libdir}/gconv ${D}${libdir}
> > -		fi
> > -		if [ -e ${LOCALETREESRC}/${datadir}/i18n ]; then
> > -			cp -R --no-dereference --
> preserve=mode,links ${LOCALETREESRC}/${datadir}/i18n ${D}${datadir}
> > -		fi
> > -	fi
> > -	if [ -e ${LOCALETREESRC}/${datadir}/locale ]; then
> > -		cp -R --no-dereference --preserve=mode,links
> ${LOCALETREESRC}/${datadir}/locale ${D}${datadir}
> > +                for d in . $(find "${LOCALETREESRC}/${libdir}/gconv"
> -type d -printf '%P ') ; do
> > +                        install -d "${D}${libdir}/gconv/$d"
> > +                        find "${LOCALETREESRC}/${libdir}/gconv/$d" -
> maxdepth 1 -type f \
> > +                        -exec install -m 0755 -t
> "${D}${libdir}/gconv/$d" {} \;
> > +                done
> > +                for d in . $(find "${LOCALETREESRC}/${datadir}/i18n"
> -type d -printf '%P ') ; do
> > +                        install -d "${D}${datadir}/i18n/$d"
> > +                        find "${LOCALETREESRC}/${datadir}/i18n/$d" -
> maxdepth 1 -type f \
> > +                        -exec install -m 0644 -t
> "${D}${datadir}/i18n/$d" {} \;
> > +                done
> >  	fi
> > -	cp -R --no-dereference --preserve=mode,links
> ${LOCALETREESRC}/SUPPORTED ${WORKDIR}
> > +        for d in . $(find "${LOCALETREESRC}/${datadir}/locale" -type
> d -printf '%P ') ; do
> > +                install -d "${D}${datadir}/locale/$d"
> > +                find "${LOCALETREESRC}/${datadir}/locale/$d" -
> maxdepth 1 -type f \
> > +                -exec install -m 0644 -t "${D}${datadir}/locale/$d"
> {} \;
> > +        done
> > +	install -m 0644 ${LOCALETREESRC}/SUPPORTED
> ${WORKDIR}/SUPPORTED
> >  }
> >
> >  inherit libc-package
> 
> The trouble is this is a workaround. The cp commands should work and
> there is some underlying issue in pseudo causing this. We really need
> to figure out what that is since this will likely just mean we see the
> problem somewhere else :(
> 
> I still suspect some kind of inode number reuse problem which these cp
> commands trigger...
> 
> Cheers,
> 
> Richard

For the record, even if I think the patch was the right thing to do, it 
did not help the situation. The first build I did after it was integrated, 
I was bit by the following:

  Failed to determine the user name of UID 323 for 
  tmp/work/core2-64-poky-linux/glibc-locale/2.29-r0/package/usr/lib/locale

where UID 323 is my UID. If you do not recognize the error message, it is 
because it comes from a local patch we have for rpm to protect us from any 
incorrect UIDs/GIDs causing incorrect RPMs to be generated (see below). I 
checked the build directory and the /usr/lib/locale directory was the only 
file or directory with an incorrect owner, which is typical of these 
pseudo related problems with incorrect owners.

[rpm will by default silently fall back to use root for any files where it 
cannot determine the name of the user or group based on its UID/GID. This 
can be due to missing information in the /etc/passwd or /etc/groups files, 
typically due to relying on another recipe in DEPENDS to create the user 
without also having an RDEPENDS on the package that creates the user, or 
because of pseudo messing up. After this bit us once where a device in 
/dev ended up in an rpm in the sstate cache with root as group instead of 
the intended video group, which in turn caused the video application to 
fail as it was no longer allowed to open its device, I created the patch. 
It is causing a fair bit of grief as it causes builds to fail randomly, 
but at least we don't end up with an incorrect sstate anymore, which is 
worse. I am not sure this patch is suitable for integration to OE-Core, 
but I have attached it if anyone is interested.] 

//Peter

-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-rpm-Make-it-an-error-if-a-UID-GID-cannot-be-turned-i.patch
Type: application/octet-stream
Size: 3889 bytes
Desc: 0001-rpm-Make-it-an-error-if-a-UID-GID-cannot-be-turned-i.patch
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20190210/63e4bb34/attachment-0001.obj>