My month for hard drive failures plus head's up about Dell's outlet

[HCHTech]

I got another notification of a failing hard drive, guess it's my month for that. This is a server that was purchased from the Dell Outlet in 2020. It came with 2 RAID cards, one running a rear-mounted SSD RAID1 (which I used for the OS) and one running a RAID10 of 8TB nearline SAS 3.5" drives. The RAID10 did not come pre-configured, that was done on the bench when we prepped the server for install.

So one of the SAS drives has failed. I open a ticket and find out that the drive that failed was not part of the original configuration of that server, and was NOT a Dell-supplied drive. It's a Seagate Constellation, I believe, ST8000NM0135. The serial number tells me it was manufactured at approximately the same time as the server, and a warranty check on Seagate's site reports that "This product was originally sold as a part of a larger system. Please contact the system manufacturer or your place of purchase for warranty support."

It looks like the original purchaser of the system I bought must have added or swapped-in (at least) this drive, and it must have come from another server built at approximately the same time. So this is apparently one of the potential downsides of getting servers from their outlet site; at least one I had not considered before.

My first inclination is not to worry too much about this, the savings over a new purchase was significantly more than the cost of a drive. But I'll certainly think more carefully about this next time I think about getting a server from the outlet site. So now I'm thinking I need to check the other drives that are still working as well.

Replacing this drive brings up another issue. That model of drive is SED. A direct replacement is about $450. A non-SED replacement is about $170. Do you think I can use a non-SED unit to replace this drive if the rest of the drives in that array are SED? I'd like to save that $ if possible, but if it doesn't work, then I should just get the direct replacement and be done. Fun times.

 

[300DDR]

I don't believe there is any reason to use a SED drive. I'd save the money.

 

[nlinecomputers]

I don't believe there is any reason to use a SED drive. I'd save the money.

Depends on the security needs of the client. You maybe required for compliance reasons to have them. If you got them the first time there must have been a reason….

 

[britechguy]

If you got them the first time there must have been a reason….

Not that I'm disagreeing with either of your other two premises, but you have to know that this one is flawed.

The amount of "accidental overkill" that occurs in hardware that just so happens to be available, or gets purchased out of ignorance, is immense.

Definitely make the inquiries about actual client need, though.

 

[nlinecomputers]

Not that I'm disagreeing with either of your other two premises, but you have to know that this one is flawed.

With respect. Hardly. Many industries have regulatory requirements for encryption. SED drive handles that without need for software like BitLocker running or as often the case Bitlocker will be enabled dependent on the SED providing the encryption. If it had an SED drive in place it is best to put one back. Not doing so will probably break things.

 

[britechguy]

@nlinecomputers

With respect, please read what I wrote.

If it's needed, then get one. If it's not, and there's a good chance it's not, then don't. It really is that simple.

Lots of hardware is not where it is because it was needed, but because it was what was available at purchase time.

 

[nlinecomputers]

And if that’s the case and the defaults were taken then the encryption is enabled. So the drive will be REQUIRED to be one. Just get the one is less work then having it wrong.

 

[HCHTech]

Just to close this loop. We bought this server from Dell's outlet store, so we took the first one that came along that had our spec requirements. This was a backup target, so we needed storage above all else. It just happened to have SED drives.

I replaced the drive with a Non-SED unit and while it did NOT report that the RAID was rebuilding and give me a status like I'm used to seeing with other Dells, it did add the right-click option on the individual drive listing in iDRAC to "cancel rebuild", so I figured it was doing it. I checked back in 24 hours and the RAID was once again healthy. The only warnings I could find about doing this was that the array would no longer be "protected" as it is when you use all SED drives. Because it was a surprise to learn that the server had at least one non-factory drive in it anyway, I would be surprised if they were all SED in the first place. Next maintenance window when I have to take the server down for some other reason, I'll take them out one at a time to snap a picture of the label of each to see, just to sate my own curiousity.

 

[HCHTech]

Trailing thought.... Because of the unusual way this drive failed, I thought I should probably run a few tests on it once I had it in hand. Then I realized you need a SAS-aware motherboard to mount the drive for testing. I don't have a spare server at-the-ready in my shop, so it's not worth the effort to try and get that going to run tests. C'est la vie!

 

[nlinecomputers]

I’m curious to see what BitLocker reports about this drive. You’ve probably decrypted the drive stack, which is probably why it took so long to rebuild it. Which if my theory is true you put the entire stack at risk because instead of just writing data to the missing disk it had to rewrite the entire stack.

 

[HCHTech]

I’m curious to see what BitLocker reports about this drive.

I'm going to invite some ire here, but I don't have Bitlocker enabled on this RAID array. Employee workstations? Yes, absolutely. Servers, not unless required for industry compliance or = medical, financial, etc., or required by the cyber-insurance vendor. The chance of data theft that Bitlocker would protect against is not worth more than the hit on R/W performance. Change my mind.

 

[nlinecomputers]

I'm going to invite some ire here, but I don't have Bitlocker enabled on this RAID array. Employee workstations? Yes, absolutely. Servers, not unless required for industry compliance or = medical, financial, etc., or required by the cyber-insurance vendor. The chance of data theft that Bitlocker would protect against is not worth more than the hit on R/W performance. Change my mind.

And I don't think that is a problem. But with SED drives unless you change defaults Bitlocker managed but running on the drives is the default. You might have been encrypted and not realized it. Even if you disable Bitlocker the drives may still be encrypted as that is what they are designed to do. The fact that you got the UNPROTECTED warning to me implies that you were encrypted and now are not.

 

[nlinecomputers]

But it could have also just been a nag notice telling you that you are not encrypted which might pop up every time you deal with the stack if encryption is possible but not enabled. A "Hey, do you really want to do that?" prompt.

 

[HCHTech]

Fair point - I 100% know that I did not have bitlocker enabled on this array, though. I set this server up from scratch, so was in control of all of it from the day I got it. I formatted the drives, setup the arrays, installed the OS, all of it. When I said I found that warning that the array would be unprotected, I meant to say I found that note online when I searched looking for consequences of replacing a single SED drive in an array with a non-SED drive. I don't honestly know if the other drives in the array are SED or not. I'll have to confirm that either in iDRAC if I can get the serial numbers or model numbers (not always, in my experience) or by physically removing the drives and recording that information from the labels.

For now, it's working, and Bitlocker is still not enabled on the storage array. It's a 6-disk RAID10 and presents as I would expect in Windows Explorer on the Host.

Just when I think I'm comfortable with most of the things I run into these days, there is always another wrinkle I haven't seen before. I guess I learned something?