Best practice two out of four raid 1 drives failing.

[Hampden Comp]

As always technibble is a tremendous resource for business owners and techs alike, thanks in advance for pointing me in the right direction. Dell Poweredge T310 server running 2008 R2. The sole role is to serve Dental Practice software called Dentrix. When I initially took the support contract I was called to replace a failing drive, it was easily identified and was replaced by Dell, resync the array and all was good. I checked in today as part of our monthly agreement and they had mentioned that the server seemed a bit pokey. Open up OSM and hardware looks all green, I check the alerts and see patrol read media errors 2272 on discs 0,1. Chatted with Dell and they will send out 2 more drives, but due to two drives and bad blocks they recommend not imaging the volume but backing up the data and setting it all up again. I do have the database backed up so if the **** hits the fan, I can grab a shovel. How would you proceed to rectify this situation with the least amount of down time. And yes, I will have the doc pony up for some decent backup software when we are in the clear.

Thanks, Kevin

 

[YeOldeStonecat]

Lets talk about the RAID setup more..and what kinda backup you have?

Do you have a RAID 1 for the OS, and a RAID 1 or 5 or 10 for the data volume ?(SQL 'n stuff of Dentrix)

Or is this just 1 big RAID volume?

What type of RAID is the 2 drives going TU on?

If it's hot swap drives....I supposed my approach would be to swap 1 drive....let it rebuild...and then when it's complete, swap the 2nd drive....let it rebuild.

Depending on the amount of data on there, size of the volume, which RAID controller you have....could do that in under an hour...or it may take hours to do each drive.

I'd then further look into possible issues..why 3 drives are reported as going TU on the server. Could be quirky firmware version on that RAID controller, or the drives themselves....something that a firmware update could address.

Probably over 25% of the hot swap drives I see flash alerts have the alerts go away simply by yanking them and slamming them back in again. SMART ain't always that smart! It cries wolf a lot.

Hope you make out OK with the remnants of blizzard Charlotte.

 

[Hampden Comp]

Thanks, My plan of attack was to rebuild one drive first and see how that looks and then pop in the other. Part of it may be the WD blue drives they are using. They are hot swap and when I pulled the first pooched drive, (which did show up on the front panel as faulty) I was floored to see a WD blue drive. I did pull one out of a 2011 iMac, so Apple is just as guilty.

I didn't set it up and it does look like one virtual volume with the OS on C and Data/sql on D. Perc6i controller and 4 disks in a single raid 1.

The drives should be here tomorrow, trying to avoid a long weekend in a Blizzard!

I have a brother over in Manchester CT that is a Manager at a software company in Hartford, he is a geek on a Harley also, an old AMF model. Thanks for your help!!!

Kevin

 

[YeOldeStonecat]

I didn't set it up and it does look like one virtual volume with the OS on C and Data/sql on D. Perc6i controller and 4 disks in a single raid 1.

sounds like a pair of RAID 1s then....
Two disks doing a RAID 1 volume for C, and two disks (typically larger) doing a RAID 1 volume for D.

I gotta head up next to Manchester next week...Willimantic actually.
Used to live up that way for a while when I went to UCONN back in the 90's...Storrs, Ashford.

 

[Hampden Comp]

Update:

So it ends up the setup is 2X500GB sata drives in a Raid 1. Dell has sent me out 2 new enterprise drives, with block errors on both drives I am assuming that a bad block on one is being covered by the other and likewise and so forth. I ran over at lunch time and connected a 1 tb usb drive. I can remote in using my Bomgar box and image the drive. Knowing that I may run into issues, I picked up a nearly identical server with the same raid controller. My goal is to take an image and restore it to my server. If that works, then I will see if the array will rebuild itself by swapping in the drives one at a time. My question is what would be the best program to image one time. I will look into storage craft as a long term solution. To save my bacon and mind, would you use the 2008r2 windows backup and what is the best way to restore it to my server.

Thanks

 

[YeOldeStonecat]

Update:

So it ends up the setup is 2X500GB sata drives in a Raid 1. Dell has sent me out 2 new enterprise drives, with block errors on both drives I am assuming that a bad block on one is being covered by the other and likewise and so forth.

That would most likely be the case...yeah. So I'd swap 1 drive....allow mirror to rebuild, and then swap the remaining drive...allow to rebuild. Done!

As for image...yeah I love StorageCraft...but costs money. For free...dunno of any I'd trust to save my bacon....I'd only want the best pay for products with support if my bacon is on the line.

Got backups of their data at least?

 

[Hampden Comp]

Crucial data is covered both with a nightly physical back up that is taken offsite and a nightly online backup. Looks like the system in question is 2008 not R2. I may try to image the virtual disc using drive image and then restore it to one disc on my server then fire the other disc in to raid it. Thanks again for your help!

 

[Tony_Scarpelli]

I agree with what has been posted if I have a relationship with the client so he knows and trusts what I am recommending different from dell.

With a new client, who has the ability to pay, I think I would stick with what Dell recommends.

In any case if the new drives and re-mirror doesn't fix the errors i would backup and do the backup and fresh install with updates and restore the data.

Another reason to do this with new clients is that you know where all the bodies are buried.