[bug #29302] Optimize checkout/update by not creating empty directories

[Julio M. Merino Vidal]

URL:
  <http://savannah.nongnu.org/bugs/?29302>

                 Summary: Optimize checkout/update by not creating empty
directories
                 Project: Concurrent Versions System
            Submitted by: jmmv
            Submitted on: Mon 22 Mar 2010 08:15:05 PM CET
                Category: None
                Severity: 3 - Normal
              Item Group: None
                  Status: None
                 Privacy: Public
             Assigned to: None
             Open/Closed: Open
                 Release: 
         Discussion Lock: Any
           Fixed Release: None
   Fixed Feature Release: None

    _______________________________________________________

Details:

When running "checkout -P" or "update -P" on a large tree, it's easy to spot
that the cleanup of empty directories at the end of the processing takes a
really long time. It is important to realize that these supposedly-empty
directories are not really empty: they also contain the CVS control directory
and their associated admin files.

The creation of these empty directories and their admin files, coupled with
the delayed removal at the end, causes the files that have to stay to end up
spread across the disk. The system cannot know what files/directories will be
deleted, and as there is a lot of disk churn due to the touching of many
files, the file system code must flush those to-be-deleted files to disk.

Seeing this behavior is really easy: e.g. one just needs to check out a fresh
copy of the pkgsrc CVS tree (see http://www.pkgsrc.org/ for the address) and
keep updating it a few times. Timing these operations across the different
runs will result in increasing numbers. Add to the equation a slow
laptop-sized hard disk and it becomes an "annoying" issue.

Some projects like the one mentioned above cannot move away from CVS anytime
soon, so they force all of their developers to be tied to the slowness of
these operations.

I would like to request an optimization to the checkout/update code: instead
of creating directories right when they are seen from the server, delay their
creation until the first non-admin file, non-directory in them gets created.
E.g. instead of doing the pruning at the end, avoid the pruning overall by not
creating the directories in the first place. This would avoid unnecessary
accesses to the file system and speed up the operation of cvs checkout/update
by a great deal.




    _______________________________________________________

Reply to this item at:

  <http://savannah.nongnu.org/bugs/?29302>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.nongnu.org/