ddrescue script

[Mindfulness]

To reduce errors in running ddrescue, I wrote a script that takes most of my human typos out of the equation. I am actually better with the command line then most linux guis. The command line works on virtually all forms of linux the same.

This following procedure works best for us. We like to backup our data recovery drives to an image file. We can have multiple images on a large 2tb drive. This script runs a preferred 2 pass method of ddrescue. The first run is a quick pass without splitting blocks or retries. the second run splits blocks and retires 3 times

To run the script, you will need to know your hard drive device name. The easiest way to get it is to run the following command twice. once before plugging in your hard drive and once after plugging in your hard drive. first we will run the following command before plugging in our drive we need to recover

root@linuxbox:~# ls /dev | grep sd  
sda
sda1
sda2
sda5
sdb
sdc
sdd
sde

next we plug in the drive, wait a minute, than run the command again

root@linuxbox:~# ls /dev | grep sd
sda
sda1
sda2
sda5
sdb
sdc
sdd
sde
sdg
sdg1
sdg2
sdg3

in this example the new drive is sdg so our drive name is /dev/sdg (it is /dev/ + sdg)

now that we know the drive name we run the following script

#!/bin/sh

#please change the value of IMAGE_DIR= to the location where you want to save your images 
IMAGE_DIR=/images

echo please enter the  device you want to clone ex. /dev/sde:
read HD_DEVICE

echo please enter a name to save your image as:
read IMAGE_NAME

echo
echo running the following command:
echo ddrescue -n $HD_DEVICE $IMAGE_DIR/$IMAGE_NAME $IMAGE_DIR/$IMAGE_NAME-log
ddrescue -n $HD_DEVICE $IMAGE_DIR/$IMAGE_NAME $IMAGE_DIR/$IMAGE_NAME-log

echo
echo running the following command:
echo ddrescue -r3 -d -f $HD_DEVICE $IMAGE_DIR/$IMAGE_NAME $IMAGE_DIR/$IMAGE_NAME-log
ddrescue -r3 -d -f $HD_DEVICE $IMAGE_DIR/$IMAGE_NAME $IMAGE_DIR/$IMAGE_NAME-log

in the script you need to set your image location one time. Every time you run the script it will ask you the device name of your hard drive to recover and the name you want to save your image as.

 

[16k_zx81]

Can you post a video of this in action?

For others interested in ddrescue / front end

http://www.bootmed.com/

 

[Mindfulness]

Can you post a video of this in action?

Hmm. Something to ponder

 

[HFultzjr]

May I butt in.

One thing I found confusing when first learning to use "ddrescue", is there are actually 2 different "programs".

The OP mentions using ddrescue.

16K's recommendation of Bootmed Plus uses dd_rescue.

Two different programs, with ddrescue being more powerful and containing more options.

I started using Bootmed Plus and it works great, but lacks many of the features that may be needed in certain situations.

I'm not knocking Bootmed Plus, it's great and is what got me started. The original suggestion of using it came from 16k in another thread.

Thank you 16k!

I finally got the hang of ddrescue, via this video.

http://www.myfixlog.com/fix.php?fid=21

The parts about using UBCD to run a live cd with ddrescue is very simple and now makes sense, at least to me. Start by using this and then learn the more powerful options that can be added.

Note......I am in no way what I would consider a viable Linux user. These are just the things that I found confusing and would like to help others avoid the same issues. dd_rescue and ddrescue use different "commands" and "parameters". ddrescue is the more powerful and useful one in my opinion.

Please.....I'm rambling on about a subject I'm just starting to comprehend. Can one of the more experienced Linux gurus cut and maybe explain in better terms what I'm trying to say.

Again, this is my limited understanding at this point. However, it's getting easier.!

From a Linux noob, that is still on the first rung of the "Linus Ladder" and a long way from the top.

I hope this is not too confusing and of some help.

 

[Larry Sabo]

Many thanks to those who post helpful tips like these!

If you can justify it, you can alternatively/also buy a hardware duplicator like the Atola Bandura. It is a stand-alone device that does what ddrescue does, and much more (duplication, diagnosis, wiping, multi-pass & reverse pass data recovery, automatic drive resets, etc.). Atola recently announced that the Bandura is being discontinued because it competes too much with their premiere data recovery product, the Atola Insight.

I just ordered one (Bandura) yesterday for $1,890 USD, Atola's clearance price. It's normally $2,495. It gets raving reviews from those who have one, and is really well suited to computer repair shops. If you have been thinking of getting a DDI or another duplicator, this might be worth considering, but be quick about it.

 

[Mindfulness]

Many thanks to those who post helpful tips like these!

If you can justify it, you can alternatively/also buy a hardware duplicator like the Atola Bandura. It is a stand-alone device that does what ddrescue does, and much more (duplication, diagnosis, wiping, multi-pass & reverse pass data recovery, automatic drive resets, etc.). Atola recently announced that the Bandura is being discontinued because it competes too much with their premiere data recovery product, the Atola Insight.

I just ordered one (Bandura) yesterday for $1,890 USD, Atola's clearance price. It's normally $2,495. It gets raving reviews from those who have one, and is really well suited to computer repair shops. If you have been thinking of getting a DDI or another duplicator, this might be worth considering, but be quick about it.

Comparing that to DDI, what are the advantages or disadvantages?

 

[Larry Sabo]

I would have preferred a DDI, but it's around $3,500 I think. Looked for a used one but they are exceedingly hard to find. The DDI does selective head reading whereas the Bandura does not. That can be handy when you have a weak head and want to recover using the good heads before swapping heads, then trying to get the data the bad head missed. If one does not do head swaps, that feature is moot. I'm sure there are other differences, like more granular control of the process, but they are only useful for those with the detailed knowledge to use them. The Bandura is aimed at non-professional data recovery techs, as it's easier to use but still very capable. If the drive requires physical recovery, take it to a DR pro. If it's logical recovery you need to do, this is what will be very effective and more practical.

 

[SAG]

I love scripting. Makes my life easier.

I hope it didn't offend you Mindful, but I added some very basic error checking.
Also, this will list the available disks for you, so no need to to peek in /dev

#!/bin/sh -e

IMAGE_DIR=/images

echo Enter the device you want to clone.
echo Available devices are: `fdisk -l 2>/dev/null | grep "Disk \/" | grep -v "\/dev\/md" | awk '{print $2}' | sed -e 's/://g'`
read HD_DEVICE

if
 [ `df | grep -q $HD_DEVICE` ] ; then
  echo $HD_DEVICE appears to be mounted.
  echo Exiting...
  exit
fi

if
 [ ! -e $HD_DEVICE ] ; then
  echo Sorry the device $HD_DEVICE does not exist
  echo Exiting...
  exit
fi

echo Enter a name to save your image as:
read IMAGE_NAME

echo
echo Running the following command:
echo ddrescue -n $HD_DEVICE $IMAGE_DIR/$IMAGE_NAME $IMAGE_DIR/$IMAGE_NAME-log
ddrescue -n $HD_DEVICE $IMAGE_DIR/$IMAGE_NAME $IMAGE_DIR/$IMAGE_NAME-log

echo
echo running the following command:
echo ddrescue -r3 -d -f $HD_DEVICE $IMAGE_DIR/$IMAGE_NAME $IMAGE_DIR/$IMAGE_NAME-log
ddrescue -r3 -d -f $HD_DEVICE $IMAGE_DIR/$IMAGE_NAME $IMAGE_DIR/$IMAGE_NAME-log

 

[lcoughey]

I would have preferred a DDI, but it's around $3,500 I think. Looked for a used one but they are exceedingly hard to find. The DDI does selective head reading whereas the Bandura does not. That can be handy when you have a weak head and want to recover using the good heads before swapping heads, then trying to get the data the bad head missed. If one does not do head swaps, that feature is moot. I'm sure there are other differences, like more granular control of the process, but they are only useful for those with the detailed knowledge to use them. The Bandura is aimed at non-professional data recovery techs, as it's easier to use but still very capable. If the drive requires physical recovery, take it to a DR pro. If it's logical recovery you need to do, this is what will be very effective and more practical.

With the use of the DRE, you can also select only the sectors associated with specific files to be mirrored, reducing the imaging time drastically if the volume of files to be recovered is significantly smaller than the size of the drive. (ie, 50GB of data vs 2TB drive). And this process still goes from sector 0 to the end, but just skips any sectors that aren't selected to be mirrored. It reduces time and saves the heads.

 

[Mindfulness]

I love scripting. Makes my life easier.

I hope it didn't offend you Mindful, but I added some very basic error checking.
Also, this will list the available disks for you, so no need to to peek in /dev

#!/bin/sh -e

IMAGE_DIR=/images

echo Enter the device you want to clone.
echo Available devices are: `fdisk -l 2>/dev/null | grep "Disk \/" | grep -v "\/dev\/md" | awk '{print $2}' | sed -e 's/://g'`
read HD_DEVICE

if
 [ `df | grep -q $HD_DEVICE` ] ; then
  echo $HD_DEVICE appears to be mounted.
  echo Exiting...
  exit
fi

if
 [ ! -e $HD_DEVICE ] ; then
  echo Sorry the device $HD_DEVICE does not exist
  echo Exiting...
  exit
fi

echo Enter a name to save your image as:
read IMAGE_NAME

echo
echo Running the following command:
echo ddrescue -n $HD_DEVICE $IMAGE_DIR/$IMAGE_NAME $IMAGE_DIR/$IMAGE_NAME-log
ddrescue -n $HD_DEVICE $IMAGE_DIR/$IMAGE_NAME $IMAGE_DIR/$IMAGE_NAME-log

echo
echo running the following command:
echo ddrescue -r3 -d -f $HD_DEVICE $IMAGE_DIR/$IMAGE_NAME $IMAGE_DIR/$IMAGE_NAME-log
ddrescue -r3 -d -f $HD_DEVICE $IMAGE_DIR/$IMAGE_NAME $IMAGE_DIR/$IMAGE_NAME-log

Thanks! Very beautiful code

 

[Larry Sabo]

With the use of the DRE, you can also select only the sectors associated with specific files to be mirrored, reducing the imaging time drastically if the volume of files to be recovered is significantly smaller than the size of the drive. (ie, 50GB of data vs 2TB drive). And this process still goes from sector 0 to the end, but just skips any sectors that aren't selected to be mirrored. It reduces time and saves the heads.

Thanks, Luke. I forgot about that. I also forgot to mention that the Bandura clones drives to other drives, it doesn't create drive image files. That's another plus for DDI (and other imagers vs drive duplicators). Given the cost difference, seems like a no-brainer to me for computer shop operators.

 

[Mindfulness]

fdisk -l 2>/dev/null

what is happening here?

root@linuxbox:~# fdisk -l 2>/dev/null | grep "Disk \/"
Disk /dev/sda: 2000.4 GB, 2000398934016 bytes
Disk /dev/sdg: 250.1 GB, 250059350016 bytes
Disk /dev/sdi: 250.1 GB, 250059350016 bytes
root@linuxbox:~# fdisk -l 2>/dev/null | grep "Disk /"
Disk /dev/sda: 2000.4 GB, 2000398934016 bytes
Disk /dev/sdg: 250.1 GB, 250059350016 bytes
Disk /dev/sdi: 250.1 GB, 250059350016 bytes

I don't understand why you did "Disk \/" instead of "Disk /". They both give the same output

 

[16k_zx81]

May I butt in.

One thing I found confusing when first learning to use "ddrescue", is there are actually 2 different "programs".

The OP mentions using ddrescue.

16K's recommendation of Bootmed Plus uses dd_rescue.

Two different programs, with ddrescue being more powerful and containing more options.

I started using Bootmed Plus and it works great, but lacks many of the features that may be needed in certain situations.

I'm not knocking Bootmed Plus, it's great and is what got me started. The original suggestion of using it came from 16k in another thread.

Thank you 16k!

I finally got the hang of ddrescue, via this video.

http://www.myfixlog.com/fix.php?fid=21

The parts about using UBCD to run a live cd with ddrescue is very simple and now makes sense, at least to me. Start by using this and then learn the more powerful options that can be added.

Note......I am in no way what I would consider a viable Linux user. These are just the things that I found confusing and would like to help others avoid the same issues. dd_rescue and ddrescue use different "commands" and "parameters". ddrescue is the more powerful and useful one in my opinion.

Please.....I'm rambling on about a subject I'm just starting to comprehend. Can one of the more experienced Linux gurus cut and maybe explain in better terms what I'm trying to say.

Again, this is my limited understanding at this point. However, it's getting easier.!

From a Linux noob, that is still on the first rung of the "Linus Ladder" and a long way from the top.

I hope this is not too confusing and of some help.

Great post!

Shame I am on tablet or would plus rep

Ok so can someone post an 'idiots guide' on how to use the modified script, above?

J

 

[SAG]

Portability.
Your /bin/sh is likely a symlink to /bin/bash, whereas that is not always the case. Some people prefer ksh, csh or other shells.
The \ simply escapes the next character, in this case /.
For bash it isn't necessary.

fdisk -l 2>/dev/null

what is happening here?

root@linuxbox:~# fdisk -l 2>/dev/null | grep "Disk \/"
Disk /dev/sda: 2000.4 GB, 2000398934016 bytes
Disk /dev/sdg: 250.1 GB, 250059350016 bytes
Disk /dev/sdi: 250.1 GB, 250059350016 bytes
root@linuxbox:~# fdisk -l 2>/dev/null | grep "Disk /"
Disk /dev/sda: 2000.4 GB, 2000398934016 bytes
Disk /dev/sdg: 250.1 GB, 250059350016 bytes
Disk /dev/sdi: 250.1 GB, 250059350016 bytes

I don't understand why you did "Disk \/" instead of "Disk /". They both give the same output

 

[HFultzjr]

Great post!

Shame I am on tablet or would plus rep

Ok so can someone post an 'idiots guide' on how to use the modified script, above?

J

J,

I think the code supplied is WAY beyond what I can do or comprehend. Probably can't be deciphered into an 'idiots guide'. If you didn't already try it, follow the suggestions in the video and do an actual test. I think you will find it very rewarding to see ddrescue copying info with just a few commands. Then go from there. That is only the very beginning of what it will do, but will lay a solid foundation of what and how it does it. Then build on that by adding other commands one at a time. Let me know if you need any help. Simple stuff of course......LOL

Harold

 

[SAG]

Ok so can someone post an 'idiots guide' on how to use the modified script, above?

IMAGE_DIR=/images
This sets the variable on where you want your output disk image stored.
You can set it to /images, or /root/images, or /mnt/NAS/images, etc.
You change this variable in the script before running it.

echo Enter the device you want to clone.
echo Available devices are: `fdisk -l 2>/dev/null | grep "Disk \/" | grep -v "\/dev\/md" | awk '{print $2}' | sed -e 's/://g'`
read HD_DEVICE
This prompts you for the linux name of the disk to clone.
It uses the 'df' command to get a list of all availabe disks connected to the system. In most cases, /dev/sda is the OS disk, so anything other than that is a good bet.
It sets the disk name you type (/dev/sdc for instance) as the HD_DEVICE variable

if
[ `df | grep -q $HD_DEVICE` ] ; then
echo $HD_DEVICE appears to be mounted.
echo Exiting...
exit
fi
This is just a little error checking. It checks to make sure the disk you are trying to clone is not currently mounted.

if
[ ! -e $HD_DEVICE ] ; then
echo Sorry the device $HD_DEVICE does not exist
echo Exiting...
exit
fi
Another little error checking. Basically I just tossed this in there to make sure you typed "/dev/sdc| correctly.

echo Enter a name to save your image as:
read IMAGE_NAME
This prompts you for the name of the image file and sets that as the variable IMAGE_NAME.

echo
echo Running the following command:
echo ddrescue -n $HD_DEVICE $IMAGE_DIR/$IMAGE_NAME $IMAGE_DIR/$IMAGE_NAME-log
ddrescue -n $HD_DEVICE $IMAGE_DIR/$IMAGE_NAME $IMAGE_DIR/$IMAGE_NAME-log

echo
echo running the following command:
echo ddrescue -r3 -d -f $HD_DEVICE $IMAGE_DIR/$IMAGE_NAME $IMAGE_DIR/$IMAGE_NAME-log
ddrescue -r3 -d -f $HD_DEVICE $IMAGE_DIR/$IMAGE_NAME $IMAGE_DIR/$IMAGE_NAME-log
This is the part that does the work. It does a double pass with ddrescue.
The first pass is a basic quick pass with no splitting.
The second pass tries to fill in any holes from bad sectors.

Google for the manpage on GNU ddrescue to find out what each flag means and all the stuff associated with it... plus there are tons more options - but his script is likely what 90% of people will be doing anyhow.

 

[Mindfulness]

fdisk -l 2>/dev/null
fdisk -l

these 2 lines of code seem to give identical output. is this also for compatibily with other shells? What does 2>/dev/null accomplish?

 

[SAG]

fdisk -l 2>/dev/null
fdisk -l

these 2 lines of code seem to give identical output. is this also for compatibily with other shells? What does 2>/dev/null accomplish?

2>/dev/null
It's a redirect of STDERR

It's not necessary, just habit forming - and good practice I suppose.

 

[Mindfulness]

Cool. That makes sense. I found a nice post explaining why to use it
http://www.jstorimer.com/blogs/workingwithcode/7766119-when-to-use-stderr-instead-of-stdout

 

[16k_zx81]

plus there are tons more options - but his script is likely what 90% of people will be doing anyhow.

Cool. Thanks

Im assuming I copy and paste the text into some sort of file, name it, and then execute it from the terminal prompt in Linux?

So...

1. Whats the best path to store the file? user directory?

2. how does one execute the file from terminal? Can this be run from a desktop shortcut?

As you can see Im not overly confident with Linux. Years ago I played with all this stuff but long forgotten now. Seems I need to renew my interest though as so many knowledgeable people are pointing to this system as #1 for data recovery.

Would appreciate any help people can offer on the simpler aspects

Thanks

J