Re: [Help-bash] xmalloc: ../../../bash/lib/sh/makepath.c:116: cannot allocate 265 bytes

[Bob Proulx]

adrelanos wrote:
> #!/bin/bash
> set -x
> shopt -s globstar
> echo /** > ~/testfile

Wow!  That will be a memory hog.  Megabytes and megabytes.  How much
virtual memory does your machine have available?

Do you have any network mounted file systems?  If so then the above
would try to expand all of those too.  That could cause a lot of
network traffic and file system activity on the remote systems too.

Do you really want all of the files printed on one line without any
newlines between them?

> echo "end"
> 
> I am getting:
> 
> ./a: xmalloc: ../../../bash/lib/sh/makepath.c:116: cannot allocate 265 bytes 
> (1123962880 bytes allocated)
> 
> Is this expected?

It depends upon how much memory you have in your system.  If you do
not have that much memory then yes this is expected.  I don't think I
own a machine with enough memory so that would be expected on all of
my systems.  If you are running a large 64-bit system with a lot of
virtual memory then your machine might be able to struggle through it.
It above has allocated 1,123,962,880 bytes.  Well over a gigabyte
already.  At that point it could not allocate any more memory.

What system are you using?  I ask because by default the Linux kernel
uses memory overcommit.  (Which I think is bad for a production
installation.  Okay for development.  But in my opinion bad for
production.)  In which case the above error wouldn't have happened.
So I assume you are running on a classic Unix machine or a system that
has set overcommit off.

> Should I better use "find" for such large requests? Can "find" be
> safely combined with "sort"?

Definitely better with find!  The 'echo /**' will attempt to do the
same thing as find but will attempt to do it all in current memory.
Then it will print out all of the files on one line.  That will be
gigabytes that will need to be kept in memory for the final print of
everything all at once.

Find it much better.  And it will put newlines between the file
names.  You probably want -xdev to avoid crossing mount points and
going down /dev and /proc and others.

  find / -xdev > ~/testfile

Rather than answer your question about combining sort with find and
about filenames with arbitrary whitespace such as newlines in them I
will pause and ask you to say what you are trying to do?

Bob