[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Commiting Large Files.

From: Derek Robert Price
Subject: Re: Commiting Large Files.
Date: Wed, 26 Mar 2003 12:27:30 -0500
User-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.2) Gecko/20030208 Netscape/7.02

Richard Horton wrote:

For various reasons we're using CVSNT to maintain a repository which amongst
other goodies contains large image files of laptop's (around 650MB).
I am aware that client.c reads the whole file (in send_modified) into
contiguous memory and so this is causing the commit to fail with out of
memory error in buffer.c.

Is there a fix or work around for this?
The images can NOT be compressed further and spliting them would lead to
problems in reassembling them (well not so much problems as an admin
overhead we can ill afford).

Any suggestions would be appreciated.

There's not really a fix or workaround, other than not checking the files in. I started rewriting some of the CVS server code to use mmap to open RCS archives for reading a year and a half ago or so. The idea is to use the system's mmap function when available to let the system hand you what looks like a memory pointer but allow the system to do the paging for you. In theory, the OS could be using the file itself as "swap" but I'm not sure it actually does. Apache uses this method when reading files as the fastest and most efficient way to go.

You could use this as a starting point to opening all files that way. I'd like to see the patch when you were done.



Email: address@hidden

Get CVS support at <http://ximbiot.com>!
The Gothic idea that we were to look backwards instead of forwards for the
improvement of the human mind, and to recur to the annals of our ancestors for
what is most perfect in government, in religion and in learning, is worthy of
those bigots in religion and government by whom it has been recommended, and
whose purposes it would answer.  But it is not an idea which this country will
                        - Thomas Jefferson to Joseph Priestley, 1800

reply via email to

[Prev in Thread] Current Thread [Next in Thread]