[OE-core] [PATCH 1/1] package_manager.py: fix for "Argument list too long"

[Robert Yang]

On 10/14/2016 05:58 PM, Robert Yang wrote:
>
>
> On 10/13/2016 06:39 PM, Burton, Ross wrote:
>>
>> On 13 October 2016 at 10:45, Robert Yang <liezhi.yang at windriver.com
>> <mailto:liezhi.yang at windriver.com>> wrote:
>>
>>     This is because "copyhardlinktree(deploy_arch_dir, arch_channel)" does:
>>     "cp -afl deploy_arch_dir/* arch_channel", while the deploy_arch_dir/* is
>>     expanded to "deploy_arch_dir/pkg1 deploy_arch_dir/pkg2
>>     deploy_arch_dir/pkg3 ..." which causes the "Argument list too long",
>>     change cwd to deploy_arch_dir can avoid the error.
>>
>>
>> Would it be better to change the implementation of copyhardlinktree so that it
>> *can't* have this problem?
>
> Good idea, thanks, updated in the repo:
>
>   git://git.openembedded.org/openembedded-core-contrib rbt/long
>   http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=rbt/long
>

Sorry, this patch is not what I wanted to paste, it should be:

diff --git a/meta/lib/oe/path.py b/meta/lib/oe/path.py
index 06a5af2..1d9cca5 100644
--- a/meta/lib/oe/path.py
+++ b/meta/lib/oe/path.py
@@ -80,15 +80,20 @@ def copyhardlinktree(src, dst):
          cmd = "cd %s; find . -type d -print | tar --xattrs 
--xattrs-include='*' -cf - -C %s -p --no-recursion --files-from - | tar --xattrs 
--xattrs-include='*' -xf - -C %s" % (src, src, dst)
          subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
          source = ''
+        oldcwd = os.getcwd()
          if os.path.isdir(src):
+            dst = os.path.realpath(dst)
+            # chdir() to src to avoid "Argument list too long" error
+            os.chdir(src)
              import glob
-            if len(glob.glob('%s/.??*' % src)) > 0:
-                source = '%s/.??* ' % src
-            source = source + '%s/*' % src
+            if len(glob.glob('./.??*')) > 0:
+                source = './.??* '
+            source += './*'
          else:
              source = src
          cmd = 'cp -afl --preserve=xattr %s %s' % (source, dst)
          subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
+        os.chdir(oldcwd)
      else:
          copytree(src, dst)

// Robert

>
> Author: Robert Yang <liezhi.yang at windriver.com>
> Date:   Thu Oct 13 01:28:41 2016 -0700
>
>     oe/path.py: fix for "Argument list too long"
>
>     Fixed when len(TMPDIR) = 410:
>     $ bitbake core-image-sato-sdk
>     [snip]
>     Subprocess output:
>     /bin/sh: /bin/cp: Argument list too long
>
>     ERROR: core-image-sato-sdk-1.0-r0 do_rootfs: Function failed: do_rootfs
>     [snip]
>
>     This is because "copyhardlinktree(src, dst)" does "cp -afl src/* dst",
>     while src/* is expanded to "src/file1 src/file2, src/file3..." which
>     causes the "Argument list too long", change cwd to src can fix the
>     problem.
>
>     Signed-off-by: Robert Yang <liezhi.yang at windriver.com>
>
>
> diff --git a/meta/lib/oe/path.py b/meta/lib/oe/path.py
> index 06a5af2..13821b3 100644
> --- a/meta/lib/oe/path.py
> +++ b/meta/lib/oe/path.py
> @@ -80,14 +80,18 @@ def copyhardlinktree(src, dst):
>          cmd = "cd %s; find . -type d -print | tar --xattrs --xattrs-include='*'
> -cf - -C %s -p --no-recursion --files-from - | tar --xattrs --xattrs-include='*'
> -xf - -C %s" % (src, src, dst)
>          subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
>          source = ''
> +        # chdir() to src to avoid "Argument list too long" error
> +        oldcwd = os.getcwd()
>          if os.path.isdir(src):
> +            os.chdir(src)
>              import glob
> -            if len(glob.glob('%s/.??*' % src)) > 0:
> -                source = '%s/.??* ' % src
> -            source = source + '%s/*' % src
> +            if len(glob.glob('.??*')) > 0:
> +                source = '.??* '
> +            source += '*'
>          else:
>              source = src
>          cmd = 'cp -afl --preserve=xattr %s %s' % (source, dst)
> +        os.chdir(oldcwd)
>          subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
>      else:
>          copytree(src, dst)
>
> // Robert
>
>>
>> Ross