Re: Problems with backup after using rdiff-backup-delete.py

[EricZolf]

Hi,

whatever Joe said (I didn't check in details but sounds reasonable), plus it 
must be clear that the scripts under misc are potentially dangerous, rather 
untested, and definitely not regularly tested, hence at this point not endorsed 
by the development team.

We apologize for the mess and will add a note to this regard before we'll look 
into improving the situation, as some of the utilities can possibly be of use.

In the meantime, @David Joe has given hints what needs to be removed to fix 
your repo (after having created a... backup of it), else let us know if you 
need more help.

KR, Eric

On November 4, 2019 9:35:41 PM UTC, Joe Steele <address@hidden> wrote:
>On 11/4/2019 2:22 PM, David Bristow wrote:
>> Hello,
>> I had to delete all references to a folder in one of my backups,
>using
>> rdiff-backup-delete.py, which one of the devs basically wrote for us.
>> The script completely removes certain files from the backup going
>back
>> to the first time it was used.  This is due to the customer requiring
>> that all copies of certain files are removed.
>> 
>> We used rdiff-backup-delete.py to delete a folder, "ibest", last
>week.
>> The folder was removed, but every back up since then has been giving
>> errors (see below this message).
>> 
>
>Interesting.  I had not heard of rdiff-backup-delete.py before.  It 
>apparently was written by Sol1 a couple years ago and was added to the 
>github repo:
>
>https://github.com/rdiff-backup/rdiff-backup/commit/858fb21806187aac09ba31286eb4d128f4613410
>
>The utility would appear to be incomplete.  It makes no attempt to
>scrub 
>the extended_attributes.*.gz files and the access_control_lists.*.gz 
>files for references to files that are to be deleted.
>
>Your error log implies that rdiff-backup was attempting to perform a 
>regression (regress.py), and in the process of doing that, it was 
>attempting to deal with extended attributes (eas_acls.py: join_ea_iter)
>
>when suddenly it could not find a file it needed to process 
>(ibest/original_data/gates_study/0506/basicskillssesa56.dta).  I'm 
>guessing that that file name came from the extended_attributes file 
>which the utility didn't scrub.
>
>Another area where the utility is deficient is its failure to look for 
>files that may have been stored under
>rdiff-backup-data/long_filename_data.
>
>The utility needs some work.
>
>--Joe
>
>
>> "ibest" was the name of the top-level folder whose contents,
>including
>> all sub-folders, totally about 250 files and about 2.4G storage, had
>> been deleted from the source. All metadata and references to these
>> folders should have been deleted by rdiff-bacup-delete.py.
>> 
>> 
>> We are running rdiff-backup-1.2.8-7.el6.x86_64 running on a RHEL box.
>> 
>> Thanks,
>> David
>> 
>>    File "/usr/lib64/python2.6/site-packages/rdiff_backup/Main.py",
>line
>> 304, in error_check_Main
>>      try: Main(arglist)
>>    File "/usr/lib64/python2.6/site-packages/rdiff_backup/Main.py",
>line
>> 324, in Main
>>      take_action(rps)
>>    File "/usr/lib64/python2.6/site-packages/rdiff_backup/Main.py",
>line
>> 280, in take_action
>>      elif action == "backup": Backup(rps[0], rps[1])
>>    File "/usr/lib64/python2.6/site-packages/rdiff_backup/Main.py",
>line
>> 337, in Backup
>>      backup_final_init(rpout)
>>    File "/usr/lib64/python2.6/site-packages/rdiff_backup/Main.py",
>line
>> 501, in backup_final_init
>>      checkdest_if_necessary(rpout)
>>    File "/usr/lib64/python2.6/site-packages/rdiff_backup/Main.py",
>line
>> 920, in checkdest_if_necessary
>>      dest_rp.conn.regress.Regress(dest_rp)
>>    File "/usr/lib64/python2.6/site-packages/rdiff_backup/regress.py",
>> line 71, in Regress
>>      for rf in iterate_meta_rfs(mirror_rp, inc_rpath): ITR(rf.index,
>rf)
>>    File "/usr/lib64/python2.6/site-packages/rdiff_backup/regress.py",
>> line 197, in iterate_meta_rfs
>>      for raw_rf, metadata_rorp in collated:
>>    File
>"/usr/lib64/python2.6/site-packages/rdiff_backup/rorpiter.py",
>> line 100, in Collate2Iters
>>      try: relem2 = riter2.next()
>>    File
>"/usr/lib64/python2.6/site-packages/rdiff_backup/eas_acls.py",
>> line 210, in join_ea_iter
>>      assert rorp, "Missing rorp for index %s" % (ea.index,)
>> Traceback (most recent call last):
>>    File "/usr/bin/rdiff-backup", line 30, in <module>
>>      rdiff_backup.Main.error_check_Main(sys.argv[1:])
>>    File "/usr/lib64/python2.6/site-packages/rdiff_backup/Main.py",
>line
>> 304, in error_check_Main
>>      try: Main(arglist)
>>    File "/usr/lib64/python2.6/site-packages/rdiff_backup/Main.py",
>line
>> 324, in Main
>>      take_action(rps)
>>    File "/usr/lib64/python2.6/site-packages/rdiff_backup/Main.py",
>line
>> 280, in take_action
>>      elif action == "backup": Backup(rps[0], rps[1])
>>    File "/usr/lib64/python2.6/site-packages/rdiff_backup/Main.py",
>line
>> 337, in Backup
>>      backup_final_init(rpout)
>>    File "/usr/lib64/python2.6/site-packages/rdiff_backup/Main.py",
>line
>> 501, in backup_final_init
>>      checkdest_if_necessary(rpout)
>>    File "/usr/lib64/python2.6/site-packages/rdiff_backup/Main.py",
>line
>> 920, in checkdest_if_necessary
>>      dest_rp.conn.regress.Regress(dest_rp)
>>    File "/usr/lib64/python2.6/site-packages/rdiff_backup/regress.py",
>> line 71, in Regress
>>      for rf in iterate_meta_rfs(mirror_rp, inc_rpath): ITR(rf.index,
>rf)
>>    File "/usr/lib64/python2.6/site-packages/rdiff_backup/regress.py",
>> line 197, in iterate_meta_rfs
>>      for raw_rf, metadata_rorp in collated:
>>    File
>"/usr/lib64/python2.6/site-packages/rdiff_backup/rorpiter.py",
>> line 100, in Collate2Iters
>>      try: relem2 = riter2.next()
>>    File
>"/usr/lib64/python2.6/site-packages/rdiff_backup/eas_acls.py",
>> line 210, in join_ea_iter
>>      assert rorp, "Missing rorp for index %s" % (ea.index,)
>> AssertionError: Missing rorp for index ('ibest', 'original_data',
>> 'gates_study', '0506', 'basicskillssesa56.dta')
>> 
>> 
>>