Re: [Bug-ddrescue] ddrescue 1.3 - questions from a newbie

[Matt Boge]

Hey guys,
 
I don't want to be a pest, but if there is any chance someone could take a few 
minutes and look at my questions below, I would definitely appreciate it.  
Again, I really appreciate what you have all done already, but I'm kinda dead 
in the water right now and I'm a little anxious about moving forward on my own.
 
I have been checking out the rescued data using Knoppix, and while a lot of 
data was saved, I noticed a huge glaring hole:  the My Pictures folder was 
missing.  As you can probably imagine, this is one of the few folders I really 
cared about rescuing and while I was expecting that some of the data might be 
lost, I certainly wasn't expecting the entire folder to not show up.
 
Anyway, I think I'd like to give SpinRite a shot on the damaged drive to see if 
I can eek out anymore data, but before I do (and possibly damage the drive 
further), I want to be sure that the stuff I did pull off already with ddrescue 
is safe.  In that vein, I want to have two copies of the rescued data: one 
scratch drive (to add any potential new data saved by SpinRite and updated by a 
second running of ddrescue) and one "safe" copy that I can revert to if my 
scratch drive fiddlings go bad.
 
Thanks for the time you have put in already, and I will certainly understand if 
you don't have the time to help me some more.  Either way, I will post follow 
up emails to update the progress... if nothing more than to bring closure to 
the thread.
 
 - Matt


----- Original Message ----
From: Matt Boge <address@hidden>
To: address@hidden
Sent: Friday, January 19, 2007 3:05:11 PM
Subject: Re: [Bug-ddrescue] ddrescue 1.3 - questions from a newbie


Hi David,

I'd like to thank you for your detailed response... I ended up moving the 
process to a secondary computer (to free up my primary one to reinstall 
Windows) and after many, many hours, I'm happy to say it finally finished!  I 
ended up with about 2000+ errors for about 250MB of my 200GB drive.  Pretty 
good, I think.

Anyway, after reading your message, I realize that I made a big mistake by 
copying the data to an unpartitioned drive.  Hopefully, I can follow your very 
helpful instructions to attempt to rescue my data.

So, here's where I am:

1. I have 3 drives connected to my computer, which I am running the Knoppix 
Live CD to access.

/dev/hda - Damaged 200 GB drive
/dev/hdb - New 250 GB drive
/dev/sda - New SATA 300 GB drive

2. I used ddrescue to copy the data to /dev/hdb.  Here was the specific command 
I used:

   ddrescue -B -n /dev/hda1 /dev/hdb rescued.log

As you've noted, I failed to copy to a partition on the new drive (i.e. 
/dev/hdb1).  Actually, I'm not really clear if I was supposed to create the 
partition first (using cfdisk?) or if the ddrescue command would have created 
it.

3. I followed your instructions and copied the MBR from the failed drive to an 
image file with the following command: 

   ddrescue -B -n -s 63 /dev/hda rescued_mbr.ima rescued_mbr.log

(I used cfdisk to verify that the /dev/hda1 partition did indeed start at 
sector 63)

4. So, here I am... rescued data on hdb and an MBR image file on my USB key.  
My next step was to follow your instructions to copy both of them to my new 
SATA drive (/dev/sda):

  # Copy the MBR / primary partition table:
  ddrescue -B rescued_mbr.ima /dev/hdc
  # Copy the first (only?) data partition:
  ddrescue -B /dev/hdb /dev/hdc1

(Actually, using sda instead of hdc, above)

But, I have a couple of questions... do I need to format the new drive and 
create a partition before I move the data over?  Should I use the formatting 
tools in Linux or would I be better off with a Windows-based tool?  Also, Ariel 
mentioned that for Windows, it would be a good idea to create a partition that 
was *exactly* the same size as the original... down to the byte.  How would you 
suggest I do that?

5. Now, assuming I have completed the above steps, what should be my next move? 
 I'd like to try to scrape more data off of the original drive if I can... 
250MB lost isn't bad, but I'm afraid that may span a lot of files, so I'd like 
to try some more.  I have SpinRite, which you have already commented on (I now 
understand how it works, I think), but can you recommend any other tools?

You have also provided some good information to try to find out which files I 
am missing, but note that since I screwed up the original copy by not copying 
the MBR, those scripts won't work with the ddrescue log file.  Any suggestions 
on what I can do to get them to work?  Also, I'm a perl newbie... what do I 
need to run these scripts?

Thanks a lot for all your help!

- Matt


----- Original Message ----
From: David Burton <address@hidden>
To: Matt Boge <address@hidden>
Sent: Monday, January 8, 2007 7:32:51 PM
Subject: Re: [Bug-ddrescue] ddrescue 1.3 - questions from a newbie


Hi Matt,


Your recovery process is obviously going rather slowly, but,
take heart, sometimes the process speeds up after it gets past
the worst error area.  Unless there are only a few errors, the
copy time is almost entirely proportional to the number of
errors, because error-free sectors copy so quickly and bad
sectors take so long to fail.  (Unfortunately, I don't know of
any way to stop the drive's very slow internal error recover
process, for a quicker first pass.)


Your strategy seems pretty good except for a couple of small
mistakes.  First, your recovery command should have been:

  ddrescue -B -n /dev/hda /dev/hdb rescued.log

instead of

  ddrescue -B -n /dev/hda1 /dev/hdb rescued.log

Unfortunately, the command you used will omit the MBR, so
there'll be no partition table on /dev/hdb.  Oops!

I do NOT recommend that you start over.  Once you've scraped
some data from a drive, it is never a good idea to throw it
away and start over.  But it would be a good idea to go ahead
and grab the part that you missed:

  ddrescue -B -n -s 63 /dev/hda rescued_mbr.ima rescued_mbr.log

(That's assuming that /dev/hda1 starts at sector 63.  You can
check that with "fdisk -lu /dev/hda".)

You're gonna have a little chore, eventually, putting the two
pieces back together... you'll have to do something like this
(assuming that /dev/hdb is a scratch drive, and /dev/hdc is the
eventual new drive):

  # Copy the MBR / primary partition table:
  ddrescue -B rescued_mbr.ima /dev/hdc
  # Copy the first (only?) data partition:
  ddrescue -B /dev/hdb /dev/hdc1


Secondly, I think you mistake how SpinRite works.  Your step #4
needs to run SpinRite on the original (failing) drive, not on
the copy.  (Note: my ddr2sr.pl "ddrescue-to-SpinRite" script
will help you target just the bad areas of the drive with
SpinRite.)

When ddrescue finishes, before you run SpinRite, the new copy
will have only two kinds of sectors:  those which contain
perfect data, and those which have nothing at all (probably
zeros, if /dev/hdb was initially all zeros).

Then, when SpinRite has finished working on the bad sectors,
you'll have two kinds of formerly-bad sectors:  Those which
were fully recovered (probably about 10% of them), and those
which were only partially recovered (the other 90%).  You can
then use ddrescue to add the (fully & partially) recovered
sectors to /dev/hdb.  But be sure to save a copy of your
rescued.log file first, because ddrescue can't tell the
difference between the fully recovered sectors and the
partially recovered sectors -- they will all read
"successfully" when ddrescue sees them.

So, after you use ddrescue to add the SpinRite-recovered
sectors to /dev/hdb, the rescued.log file will no longer be
an accurate record of which parts if the drive are damaged.
(Dd-rescue's logfile has no mechanism for representing
damaged/partially-recovered sectors.)  But the combination of
the new and old ddrescue log files, plus the SpinRite logfile,
encapsulates that information.  My ddr2nfi.pl tool uses all
three logfiles to generate NFI commands for the truly damaged
sectors, for trying to deduce which files are impacted by
the damage.  (Note: run the resulting .bat file of NFI
commands on a scratch copy of the recovered drive, not the
main/best copy, since Windows always "messes with" a drive
when it mounts it.)

For hours of entertainment, download my Perl scripts that
I use when doing dd-rescue recoveries.  One

http://www.burtonsys.com/download/ddr2sr.zip

It contains:

  clustersize.pl      -- "Shows cluster size(s) for FAT and NTFS partitions"
  ddrsummarize.pl     -- "Summarize a ddrescue log file"
  ddrsplit.pl         -- "DDRescue logfile SPLITter"
  ddrcombine.pl       -- "DDRescue logfile COMBINEr"
  ddrlogand.pl        -- "DDR LOGical .AND."
  ddrlogor.pl         -- "DDR LOGical .OR."
  ddrlognot.pl        -- "DDR LOGical .NOT."
  ddr2sr.pl           -- "ddrescue-to-SpinRite"
  ddr2nfi.pl          -- "DDRescue to NFI"
  nficruncher.pl      -- "Process the output of many NFI commands"
  fat32inf.pl         -- "FAT32 info (this one is still under construction)
  foreach.pl          -- "Repeat a command for each filename or argument"
  ddrcopyincrement.pl -- "Copy additional recovered sectors"
  ddrwipe.pl          -- "Obliterate all the GOOD sectors on a failing disk 
drive"
  samplescripts.zip   -- some shell scripts
  ddrescue_perl_helper_programs.txt  -- documentation (see also comments in the 
scripts)

There are examples in the samplescripts.zip of using a
"raw" device for a later "pass" to read sector-at-a-time,
so that the unrecovered parts will be down to a granularity
of one sector, instead of a full Linux buffered disk block.
Unfortunately, raw devices vary in implementation between
Linux versions, so you might have to fiddle with the scripts
to make that part work.

Unfortunately, some of the other scripts assume that the
destination is a copy of the full drive, including MBR, and
expect the logfile to describe a whole drive.  After you put
the two recovered pieces back together, your rescued.log file
will not properly describe the "holes" in your entire /dev/hdc
drive, since all the LBA sector numbers will be off by
(probably) 63.  But you could fix it pretty easily with a
little (Perl?) program that runs through the logfile and fixes
up the sector offsets.


For your step #5 (which seems optimistic, at this point, due
to the apparently rather severe damage to that drive), if the
XP registry files are damaged, might need to recover a good
copy of the registry from the system restore points.  Contact
me for the .bat script I've written to do that, if you get
that far.


For your step #6 (which seems more realistic for this drive),
you can use xxcopy (a very nice shareware tool), and a command
like this:

REM   xxcopy_only_missing_files.bat
REM
REM   Copies from source (%%1) to destination (%%2) directory tree
REM   only those files which do not already exist in the destination
REM   directory tree, except does not copy empty directories.
REM
REM   Example:
REM      xxcopy_only_missing_files.bat H:\ I:\
REM
xxcopy %1 %2 /H /K /BB /S /Q /ZE

Get xxcopy here:

http://www.xxcopy.com/


For your step #7, Microsoft's "nfi" utility is your friend
(assuming that it is an NTFS file system).  Two of the Perl
scripts in my collection are helpers for this chore:

  ddr2nfi.pl          -- "DDRescue to NFI"
  nficruncher.pl      -- "Process the output of many NFI commands"

Download NFI from Microsoft's web site:
http://www.google.com/search?q=%22ntfs+file+sector+information%22+site%3Amicrosoft.com

A few months ago I posted some other notes on this process here:
http://www.linuxquestions.org/questions/showthread.php?s=2156247cf74a1f0b7e8a0b3b58400dd0&p=2439434#post2439434
(The question asked was about the Linux ReiserFS file system,
but, silly me, I didn't read it carefully, and posted an answer
that included instructions for NTFS data recovery.)


As for your step #9, just forget it.  That drive's toast.
Get what data you can from it, and when you are done just
junk it.  (Or sell the PCB from it on eBay... someone might
need it for a drive with a fried PCB.)


Good luck!

-Dave Burton      geek at burtonsys.com but please no spam
Geeks Alive Computer Rescue:  http://www.geeksalive.com/
PO Box 4157, Cary, NC 27519-4157 USA
Tel: 1-919-481-2183  Cell: 1-919-244-3316



address@hidden (Matt Boge) wrote:

> Hello,
>
> I ran across your tool while searching for solutions to rescue
> data from my failing hard drive.  I am a newbie to Linux, but I
> am technical and can understand and follow directions.
>
> Anyway, my 200GB NTFS failed to boot the other day, with
> Windows XP telling me it couldn't read a required file.
> Subsequent tests, specifically the Maxtor utility, told me that
> the drive was failing and I needed to replace it.  I attempted
> to repair with the Windows installation disk, but it failed,
> citing read-errors on the disk.  This is when the panic began
> to set in...
>
> I looked around for some tools that I remembered would do a
> bit->bit copy and if it ran into errors, would ignore them and
> move on.  I wanted as much data as I could get to be on a good
> drive as soon possible before the drive really kicked the
> bucket and I was hoping that I could clone the disk, have
> Windows repair the bad file and not have to reinstall all my
> apps.  I looked at Ghost and some others, but settled on
> Acronis True Image, as it seemed to do what I wanted.
> Unfortunately, it appears that True Image works best on a
> healthy drive and I was never able to get the image copied (the
> status bar never moved...  even after 24 hours).  I thought
> perhaps since the drive I was copying from was IDE and the new
> one was SATA it could have been the problem.  So, I bought a
> 250 GB IDE drive and it still didn't work.  Now, I was truly
> worried about just getting the data off of it.
>
> After some more searching I found your utility from the article
> I found on this page:
> http://www.cgsecurity.org/wiki/Damaged_Hard_Disk
>
> I'm pretty inexperienced with Linux, but I have dabbled a
> little (upgrading my TiVos, for example) and your tool looked
> like exactly what I needed.  I downloaded Knoppix, managed to
> install ddrescue from a floppy (it wasn't included in Knoppix)
> and -- after realizing I needed to be logged in as 'root' --
> ran the following command from a Konsole terminal window:
>
> ddrescue -B -n /dev/hda1 /dev/hdb rescued.log
>
> (This was the recommended command from the above guide to "grab
> most of the error-free areas in a hurry")
>
> That was about 24 hours ago and while I suppose "in a hurry" is
> a relative term, as I type this, here is what the current
> console status looks like:
>
> Current status
> rescued: 5742 MiB,   errsize: 29612 KiB,   current rate: 2304 KiB/s
>    ipos: 5771 MiB,    errors:   595,       average rate:  140 KiB/s
>    opos: 5771 MiB
> Copying data....
>
> Does that look about right?  Seems to me at this rate (approx
> 5GB/day) it's going to take about a month to complete...  am I
> reading this wrong?
>
> So I guess my first question would be did I use the correct
> command to copy over the good data quickly?
> If not, what should I be doing?
>
> Secondly, here is my attack plan...  I would appreciate it if
> someone would tell me if it looks OK:
>
> 1.  Copy the good data to a brand new HD (Copy1) using
> ddrescue.  <-- This is where I am now.
>
> 2.  Attempt to copy from the bad sectors using ddrescue.
>
> 3.  Remove the failing HD and put aside.
>
> 3.  Make another copy (Copy 2) from the first copy to another
> HD (Copy 3) -- I'm assuming it would be OK if this new HD is
> SATA, right?
>
> 4.  Use SpinRite (another recommended utility) to attempt to
> repair the bad sectors on Copy 3.
>
> 5.  Attempt to repair WindowsXP with the repair utility on the
> install CD (again, on Copy 3.  In my perfect world, Windows
> would repair successfully and I could find out which of my data
> files couldn't be repaired and go from there.)
>
> 6.  If that won't work, I would install XP on a brand new drive
> and try to copy the data files from Copy3 to the new install.
>
> 7.  Identify what files didn't make it -- not sure how I'm
> going to do that yet, but I'll cross that bridge when I come to
> it.
>
> 8.  Attempt to rescue those files specifically -- I'm not sure
> if I should try off of the original disk or from one of the
> copies.
>
> 9.  Once I'm satisfied I tried everything I could to get all
> the files I need...  I will probably run SpinRite on the
> original failed drive and see what happens.
>
> Does this seem like a prudent course?  What am I missing?
>
> Thanks for all the help, guys...  and just a quick reminder
> while you're reading this: DO A BACKUP RIGHT NOW!
>
> - Matt B


_______________________________________________
Bug-ddrescue mailing list
address@hidden
http://lists.gnu.org/mailman/listinfo/bug-ddrescue