[bitbake-devel] Filename too long

[Robert Yang]

Hi Mark,

We have a local patch to fix the problem, I had sent it to bitbake-devel
before, but not merged:

https://patchwork.openembedded.org/patch/61321/

Here is the updated patch in our bitbake:

commit ebb1190c01a15fc6e91c9810c46313c3b3fb305c
Author: Robert Yang <liezhi.yang at windriver.com>
Date:   Fri Nov 8 23:32:11 2013 +0800

     fetch2/git.py: fix the filename too long error

     When we use the PREMIRROR/MIRROR from the local disk, and if it is in a
     deep directory, for example, when len(path) > 255 (The NAME_MAX), we
     will get the "filename too long error".

     This is becuase we use ud.path.replace('/', '.') to change the path to
     the filename. We seldom see such a long url, but it is easy to produce
     it on the local machine.

     Another problem is that:

     file:///local/path

     will be converted into .local.path which is a hidden file by default.

     Don't convert the name when ud.proto == "file" will fix the problem.

     Signed-off-by: Robert Yang <liezhi.yang at windriver.com>

diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py
index f7a0c01..cc17e5a 100644
--- a/lib/bb/fetch2/git.py
+++ b/lib/bb/fetch2/git.py
@@ -195,7 +195,10 @@ class Git(FetchMethod):
                      ud.unresolvedrev[name] = ud.revisions[name]
                  ud.revisions[name] = self.latest_revision(ud, d, name)

-        gitsrcname = '%s%s' % (ud.host.replace(':', '.'), ud.path.replace('/', 
'.').replace('*', '.'))
+        if ud.proto == "file":
+            gitsrcname = '%s%s' % ("file...", os.path.basename(ud.path))
+        else:
+            gitsrcname = '%s%s' % (ud.host.replace(':', '.'), 
ud.path.replace('/', '.').replace('*', '.'))
          if gitsrcname.startswith('.'):
              gitsrcname = gitsrcname[1:]


// Robert

On 01/19/2017 03:02 AM, Mark Hatle wrote:
> I found a problem today working through a problem report.
>
> We have a situation where we automatically add certain project directories as
> premirrors within the environment.
>
> During the fetch2 work, the system takes this long pathname, and then embeds it
> into ud.lockfile parameter, we end up with something like:
>
> The path to the build/download dir is about 250 characters... (abbreviated below)
>
> ud.lockfile =
> /my/really/log/path/to/my/project/build/download/git2/file....my.really.log.path.to.my.project.layers.wr-kernel.recipes-kernel.linux.......git.kernel-4.8.x.git.lock
>
> This triggers an exception, IOError with the errno '36', and error string of
> 'File name too long'.  (Note, this is the filename, NOT the path being too long!)
>
> In the bitbake/lib/bb/utils.py: def lockfile(name, shared=False, retry=True,
> block=False)
>
> The system tries to:
>
> while True:
>     try:
>         lf = open(name, 'a+')
>         ... more work here ...
>     except Exception:
>         ... close the file, if opened ...
>         pass
>     if not retry:
>         return None
>
> This triggers an endless loop if this happens.
>
> So at a minimum we probably need to extend the exception handling to look for
> file (or path) too long errors and return/stop.
>
> I'm wondering if, something like the following should be added:
>
>  exception IOError as e:
>      errno, strerror = e.args
>      logger.fatal("IOError on lock: [%s] %s" % (errno, strerror))
>      try:
>          lf.close()
>      except Exception:
>          pass
>      retry = False
>      pass
>
> (Is stopping on all IOErrors too broad of a check?)
>
>
> Second, I'm not sure where the lockfile name is determined, but should there be
> a way in there to shorten the lockfile to say the 'basename' instead of a full
> path (if specified)?
>
> Any suggestions would be appreciated.
>
> --Mark
>