[OE-core] [PATCH 1/1] sstate.bbclass: fix parallel building issue

[Martin Jansa]

On Wed, Aug 14, 2013 at 01:28:53PM +0800, Rongqing Li wrote:
> 
> 
> On 08/14/2013 03:02 AM, Saul Wold wrote:
> > On 08/13/2013 01:20 AM, rongqing.li at windriver.com wrote:
> >> From: "Roy.Li" <rongqing.li at windriver.com>
> >>
> >> sstate_package creates hardlink from sysroot to SSTATE_BUILDDIR, then
> >> sstate_create_package will store SSTATE_BUILDDIR into a archive file by
> >> tar, but once other packages install the same file into sysroot, the
> >> creating the archive file will fail with below error:
> >>
> >>      DEBUG: Executing shell function sstate_create_package
> >>      tar: x86_64-linux/usr/share/aclocal/xorg-macros.m4: file changed
> >> as we read it
> >>
> >> This kind of error is harmless, use --ignore-failed-read to ignore it.
> >>
> > I am not sure it's so harmless, what if the file is corrupted, then we
> > have a bad sstate tarball.  You have identified the part of the root
> > cause being the hardlink, but what if the file actually does change
> > (which would be a different bug potentially), then your packaging a
> > differet set of macros (in this case) with the sysroot.
> >
> >
> > Sau!
> 
> 
> The file is not corrupted, and the file content is not changed,  "tar"
> said xorg-macros.m4 file is changed, since the number of links of
> xorg-macros.m4 has changed when other packages is doing configuration
> and call autotools_copy_aclocal to make a hardlink to ${ACLOCALDIR}
> 
> If this fix can be accepted, I will rework the commit header.

I think there is still some other issue.

I haven't seen this on ext4 filesystems, but with reiserfs I was able to
reproduce "cp: will not create hard link" issue, e.g.:

do_populate_lic_setscene task failing in sstate_install with 
cp: will not create hard link `/OE/deploy/licenses/recipe' to directory `/OE/deploy/licenses/recipe' (same path)

or
ERROR: Error executing a python function in pn.bb:
CalledProcessError: Command 'cp -afl /OE/work/armv7a-vfp-neon-oe-linux-gnueabi/pn/1.0/pkgdata/* /OE/pkgdata/armv7a-vfp-neon-oe-linux-gnueabi' returned non-zero exit status 1 with output 
cp: warning: source file `/OE/work/armv7a-vfp-neon-oe-linux-gnueabi/pn/1.0/pkgdata/pn' specified more than once

cp: will not create hard link `/OE/pkgdata/armv7a-vfp-neon-oe-linux-gnueabi/runtime' to directory `/OE/pkgdata/armv7a-vfp-neon-oe-linux-gnueabi/runtime'
cp: will not create hard link `/OE/pkgdata/armv7a-vfp-neon-oe-linux-gnueabi/runtime-reverse' to directory `/OE/pkgdata/armv7a-vfp-neon-oe-linux-gnueabi/runtime-reverse'
cp: will not create hard link `/OE/pkgdata/armv7a-vfp-neon-oe-linux-gnueabi/shlibs' to directory `/OE/pkgdata/armv7a-vfp-neon-oe-linux-gnueabi/shlibs'

Number of hardlinks is:
$ find pn/1.0/pkgdata -printf "%f/%n/%i\n"
pkgdata/5/190867045
runtime-reverse/2/190867046
pn-dbg/1/190867047
pn-dev/1/190867048
pn-doc/1/190867049
pn/1/190867067
pn-staticdev/1/190867051
pn-locale/1/190867078
runtime/2/190867053
pn-dbg.packaged/1/190867054
pn-dev.packaged/1/190867056
pn-dbg/1/190867057
pn-dev/1/190867058
pn-doc/1/190867059
pn/1/190867060
pn-staticdev/1/190867062
pn.packaged/1/190867063
pn-locale/1/190867064
pn/1/190867065
shlibs/2/190867069

find ~ -xdev -samefile pn/1.0/pkgdata 2>/dev/null
pn/1.0/pkgdata

I'm not sure where the other pkgdata hardlinks came from.

The problem is that I can reproduce it on 1-2 random recipes from few hundreds
included in bigger image and even not in every build. After the error is shown
it all looks sane, only way to manually reproduce the same error is to really
specify source dirs twice:

$ cp -afl \
/OE/work/armv7a-vfp-neon-oe-linux-gnueabi/pn/1.0/pkgdata/* \
/OE/work/armv7a-vfp-neon-oe-linux-gnueabi/pn/1.0/pkgdata/* \
/OE/pkgdata/armv7a-vfp-neon-oe-linux-gnueabi

shows exactly the same 1 warning and 3 errors.

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa at gmail.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.openembedded.org/pipermail/openembedded-core/attachments/20130814/341e2251/attachment-0002.sig>