rdiff-backup-users
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[rdiff-backup-users] rdiff-backup error / recovery failure


From: Gary Rickert
Subject: [rdiff-backup-users] rdiff-backup error / recovery failure
Date: Mon, 25 Mar 2013 10:47:55 -0600

We have been running rdiff-backup via a cron script (follows) for the last few months against 3 systems with no problem. The backup directory is rotated each week, and last night the initial (full) backup apparently failed.

BACKUP SCRIPT:
#!/bin/sh
logfile=/var/log/rdiff-backup.log
export update=$(date +'%H%M%S')
/bin/date +'%H%M%S' >> /mySQLbackup/rdiff-backup-time.log
#echo "$update" >> /var/log/rdiff-backup.log
exec >> $logfile 2>&1
export WEEK=$(date +'%V')
### The WEEK var is to recycle the backup each week.
### The time wrapper captures the Elapsed time, User time, CPU time, and the percentage of CPU time used by the process.

/usr/bin/time -f "VREI E=%E, U=%U, S=%S, P=%P" /usr/bin/rdiff-backup --remote-schema "ssh -p30000 %s /usr/bin/rdiff-backup --server" address@hidden::/var/www/apps/pfa2/shared/system /mySQLbackup/VREI/"$WEEK"-system-rdiff-backup 2>> /mySQLbackup/rdiff-backup-time.log;


LOG:
235901
TEST E=0:11.92, U=8.13, S=1.19, P=78%
VREI E=0:18.99, U=10.51, S=0.86, P=59%
THOM E=0:05.03, U=1.75, S=0.30, P=40%

It appears that the above cycle (235901) went OK. When the directory cycled it looks to have failed running the VREI cycle but no error recorded, just stopped after the 000101 cycle.

000101
TEST E=0:19.36, U=1.06, S=0.50, P=8%

Then when the next cycle (000301) started against VREI, the following error was generated, and repeated until addressed:

000301
TEST E=0:05.76, U=0.37, S=0.06, P=7%
Found interrupted initial backup. Removing...
Exception '[Errno 39] Directory not empty: '/mySQLbackup/VREI01/13-system-rdiff-backup/rdiff-backup-da
ta/increments/documents'' raised of class 'exceptions.OSError':
  File "/usr/lib64/python2.4/site-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/lib64/python2.4/site-packages/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/lib64/python2.4/site-packages/rdiff_backup/Main.py", line 280, in take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/lib64/python2.4/site-packages/rdiff_backup/Main.py", line 333, in Backup
    backup_set_rbdir(rpin, rpout)
  File "/usr/lib64/python2.4/site-packages/rdiff_backup/Main.py", line 452, in backup_set_rbdir
    fix_failed_initial_backup()
  File "/usr/lib64/python2.4/site-packages/rdiff_backup/Main.py", line 411, in fix_failed_initial_back
up
    rp.conn.rpath.delete_dir_no_files(rp)
  File "/usr/lib64/python2.4/site-packages/rdiff_backup/rpath.py", line 397, in delete_dir_no_files
    rp.delete()
  File "/usr/lib64/python2.4/site-packages/rdiff_backup/rpath.py", line 1070, in delete
    self.conn.shutil.rmtree(self.path)
  File "/usr/lib64/python2.4/shutil.py", line 163, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "/usr/lib64/python2.4/shutil.py", line 172, in rmtree
    onerror(os.rmdir, path, sys.exc_info())
  File "/usr/lib64/python2.4/shutil.py", line 170, in rmtree
    os.rmdir(path)

Traceback (most recent call last):
  File "/usr/bin/rdiff-backup", line 30, in ?
    rdiff_backup.Main.error_check_Main(sys.argv[1:])
  File "/usr/lib64/python2.4/site-packages/rdiff_backup/Main.py", line 304, in error_check_Main
    try: Main(arglist)
  File "/usr/lib64/python2.4/site-packages/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/lib64/python2.4/site-packages/rdiff_backup/Main.py", line 280, in take_action
    elif action == "backup": Backup(rps[0], rps[1])
  File "/usr/lib64/python2.4/site-packages/rdiff_backup/Main.py", line 333, in Backup
    backup_set_rbdir(rpin, rpout)
  File "/usr/lib64/python2.4/site-packages/rdiff_backup/Main.py", line 452, in backup_set_rbdir
    fix_failed_initial_backup()
  File "/usr/lib64/python2.4/site-packages/rdiff_backup/Main.py", line 411, in fix_failed_initial_back
up
    rp.conn.rpath.delete_dir_no_files(rp)
  File "/usr/lib64/python2.4/site-packages/rdiff_backup/rpath.py", line 397, in delete_dir_no_files
    rp.delete()
  File "/usr/lib64/python2.4/site-packages/rdiff_backup/rpath.py", line 1070, in delete
    self.conn.shutil.rmtree(self.path)
  File "/usr/lib64/python2.4/shutil.py", line 163, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "/usr/lib64/python2.4/shutil.py", line 172, in rmtree
    onerror(os.rmdir, path, sys.exc_info())
  File "/usr/lib64/python2.4/shutil.py", line 170, in rmtree
    os.rmdir(path)
OSError: [Errno 39] Directory not empty: '/mySQLbackup/VREI01/13-system-rdiff-backup/rdiff-backup-data
/increments/documents'
Fatal Error: Lost connection to the remote system
Command exited with non-zero status 1
VREI E=0:03.21, U=0.15, S=0.16, P=9%
THOM E=0:07.81, U=0.43, S=0.19, P=7%
000501
TEST E=0:04.31, U=0.38, S=0.08, P=10%
Found interrupted initial backup. Removing...
Fatal Error: Bad rdiff-backup-data dir on destination side

The rdiff-backup data directory
/mySQLbackup/VREI01/13-system-rdiff-backup/rdiff-backup-data
exists, but we cannot find a valid current_mirror marker.  You can
avoid this message by removing the rdiff-backup-data directory;
however any data in it will be lost.

Probably this error was caused because the first rdiff-backup session
into a new directory failed.  If this is the case it is safe to delete
the rdiff-backup-data directory because there is no important
information in it.

Fatal Error: Lost connection to the remote system
Command exited with non-zero status 1

We then attempted to rm the problem directory so the backup would regenerate. The same error was generated.

We then stopped cron and waited until the rdiff-backup activities closed, removed the directory again, and started cron. The next attempt also failed.

Cron was stopped again, the script modified to place the backup in a different directory, thinking that would assure us that there was nothing to be seen to generate the error. The first pass to that directory also errored, but then appeared to recover. Is there something dropped on the data source system that rdiff-backup would detect as a "Found interrupted initial backup."
Any other ideas of what the problem could be?

reply via email to

[Prev in Thread] Current Thread [Next in Thread]