Re: File Size Limit Exceeded G++ Linux

[Paul Pluzhnikov]

Robert Heller <heller@deepsoft.com> writes:

>   "Peter Nolan" <peter@peternolan.com>,
>   In a message on 14 Apr 2006 07:10:59 -0700, wrote :
>
> "N> Hi All,
> "N> I am running some C++ I wrote having been compiled in G++. When writing
> "N> to a file I get 'file size limit exceeded' as a message when the file
> "N> size gets to 2,147,483,647.

Are you using 'std::fstream', 'FILE*', or 'int fd' to write these files?

Using 'fstream', I get the same behavior as you do, regardless
of whether I use '-D_FILE_OFFSET_BITS=64'.

Using 'FILE *', I get "File size limit exceeded" without
-D_FILE_OFFSET_BITS=64, but can write 4GB file when compiled with
-D_FILE_OFFSET_BITS=64.

[In the process of experimentation, I have corrupted my old reiserfs
/home filesystem, which is why my reply is 2 days late ;-[

Since 'fstream::write' calls 'fwrite' to do the actual writing,
the problem must be in how libstdc++ was compiled.

In fact, looking at strace output, I observe:

  // using std::fstream
  open("junk.out", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3

  // using fopen()/fwrite()
  open("junk.out", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3

Running under debugger, I see that std::fstream version calls
fopen@GLIBC_2.1, while the fopen()/fwrite() version calls fopen64().

Looking further into gcc-3.4.0/libstdc++ sources, I see this:

  __basic_file<char>::open(const char* __name ...
#ifdef _GLIBCXX_USE_LFS
        if ((_M_cfile = fopen64(__name, __c_mode)))
#else
        if ((_M_cfile = fopen(__name, __c_mode)))
#endif

So, to solve the problem, you must rebuild libstdc++ with
-D_GLIBCXX_USE_LFS. It appears, that newer versions of g++ (at
least 3.4.0) automatically check for and enable this, but the
default g++ 3.3.3 (as shipped with Fedora Core 2) either didn't,
or was compiled on a non-LFS filesystem.

> Exactly what version of red hat linux are you using?  Which kernel
> version?  Which GLibc version?

This is likely irrelevant -- other programs *can* write large files:

> "N> We have tried creating larger
> "N> files with other programs and they work ok

which means his kernel and glibc are perfectly fine.

Cheers,
-- 
In order to understand recursion you must first understand recursion.
Remove /-nsp/ for email.