[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
diff (2.8.1) cannot work with really large files
From: |
Darren Reed |
Subject: |
diff (2.8.1) cannot work with really large files |
Date: |
Sat, 26 Jul 2008 11:21:22 -0700 |
User-agent: |
Thunderbird 2.0.0.0 (Windows/20070326) |
There is some brain dead logic in diff that seems to assume that
whichever files are being diff'd need to be read entirely into memory,
for which not even --large-spare-files fixes.
I have two 300MB text files and if I run diff on them, I get:
diff: memory exhausted
What poppycock.
I'm equally surprised that it doesn't appear to try mmap either.
Maybe we need:
--small-buffers-for-really-large-files
But I suspect that's not enough and that core parts of diff need
to be rewritten if it is going to assume that the entire file isn't in
memory.
Meanwhile, I think I'll write my own diff program that doesn't
need to allocate 600MB of RAM to tell me 5 lines are different.
- diff (2.8.1) cannot work with really large files,
Darren Reed <=