Re: mv fails with "Too many open files" when moving _many_ files

[Bob Proulx]

Cyril Bouthors wrote:
> I'm having problems when moving everything from a directory that
> contains 10852 top level directories and a total of ~300000+ files and
> subdirectories with this command:
> 
> # mv * ../../../webalizer-clients/
> 
> It fails with:
> 
> mv: cannot create regular file 
> `../../../webalizer-clients/bouthors/usage_200305.html': Too many open files

Thanks for the report.  It is much appreciated.  However I am unable
to reproduce the problem locally.  I tried this to do so:

  mkdir /tmp/foo /tmp/bar
  cd /tmp/foo
  for f in $(seq 1 30000);do touch $f;done
  mv * ../bar/

I tried both 5.2.1 and the latest 5.3.0 release image.  You said you
were using linux 2.6.6.  What glibc version are you using?

However, whenever you are moving a very large or unbounded number of
files using a shell expanded glob character such as '*' you should be
aware of exceeding the ARG_MAX kernel limitation.  For example in my
above case just removing that list of files will trigger the problem
on my system.

  rm ./*
  -bash: /bin/rm: Argument list too long

Let me suggest the following as an improvement.

  ls | xargs mv --target-directory=../../../webalizer-clients/

This works if the files all have normal names.  For handling files
with unusual characters it is always best to use null terminated
strings.  For this 'find' is best.  I tend to use the short options
but for clarity this following example has fully spelled out options
suitable for a script.

  find . -maxdepth 1 -not -path . -print0 \
    | xargs --no-run-if-empty --null mv --target-directory=/SOME/DIR/

Bob