Cannot access Domain Controller after hard drive failure

[Nerm]

Got a strange one here. I have a client that has a single domain controller environment (Server 2008 R2) so no backup domain controllers or replication partners of any kind. I know this is bad but they refused to include more than one server in their budget.

The problem is that over the weekend they had a hard drive fail in their domain controller. Swapped the drive with spare, let the raid rebuild, and when it booted back up to Server 2008 I went to login like normal and got the following error "the security database on the server does not have a computer account for this workstation trust relationship". After going through google and tons of Microsoft articles I have found all kinds of suggestions for resetting admin password with netdom, etc. However the problem lies in that all these fixes seem to require a multi-replication server environment which this client does not have. To top it all off they have not been doing system state backups either.

Monday's Suck!

 

[YeOldeStonecat]

Fun one for a Monday....I just kicked off an SBS migration, where the source server is very wobbly..on its last legs.

Anyways...lets go back to the RAID....what kind of RAID? And do you have that original drive you removed around still?

 

[Nerm]

It is a RAID5 with 3 drives plus a hot spare. Yes, I still have the original drive but it is dead as a door nail.

 

[AdamsAPlus]

So with the RAID5, So swapping and rebuilding the array shouldn't have affected it since there shouldn't have been any data affected. So let me ask a question and I'm not trying to attack your intelligence. Was the computer still running and working on the domain before you swapped the hard drive, after the old one failed?

I'm guessing you know this about a RAID5 but I'm including this just in case you don't. A RAID5 should still work with 1 bad drive.

I'm just trying to rule out the possibility that there were more than 1 bad drive.

 

[Nerm]

After a little more prodding of the client I have a little more detail of exactly what happened.

This client is somewhat "technical" so tends to tinker with things before calling me. When the server started beeping that a drive in raid had failed he powered off, pulled the failed drive, and switched over to the hot swap drive. He then proceeded to power it back on and said it was taking a long time to boot and sat at "applying settings", etc when the OS was booting. Once he got to the login screen he logged in (no errors at that point), and because the system seemed to be running slowly he immediately shut the server back off after getting logged in. (At that point I actually went silent after what I just heard)...Anyways he then puts everything back the way it was and calls me.

 

[AdamsAPlus]

oooo. Got to love it when you have technical customers.

 

[Nerm]

Yea they are wonderful aren't they? lol


With less than 25 workstations on the network I believe I am just going to wipe and reload from scratch. They have backups of their actual data so at least that part is good.

 

[Markverhyden]

Yea they are wonderful aren't they? lol


With less than 25 workstations on the network I believe I am just going to wipe and reload from scratch. They have backups of their actual data so at least that part is good.

Hopefully that is T&M. You want it to be a lesson he will not soon forget.

 

[YeOldeStonecat]

When the server started beeping that a drive in raid had failed he powered off, pulled the failed drive, and switched over to the hot swap drive. He then proceeded to power it back on and said it was taking a long time to boot and sat at "applying settings", etc when the OS was booting. (

I guess he missed the whole point of "hot swap" drives...no need to power off the server.

Small network...just 1x server...1x domain controller, common enough.
As others said....RAID 5...1x failed drive should not result in any lost data.

If the person likes to monkey around, I'd check some other things.
*event viewer
*Run dcdiag on it at all?
*Check network settings..double checking DNS

RAID 5 for the system volume? Usually RAID 5 is for a data volume. Double check that? Typically system volume is RAID 1.

No backups? How about previous versions?

 

[Nerm]

dcdiag basically said the domain did not exist.

They just had one large single volume in RAID5. They had backups but only of their data (eg. quickbooks, documents, PMS, etc).

As an update it was decided to rebuild from scratch so I spent 14 hours yesterday rebuilding the server from scratch and then rebuilt their VM of their RDS server of which they did not have backups of either (honestly though being used only for RDS that isn't that big of a deal). Today I get to rejoin computers and migrate users to the new rebuilt domain.

EDIT: They only have 25 workstations but they are spread out over 3 locations so some travel is involved.

 

[ohio_grad_06]

Maybe create an image of the server after things are set back up and keep it on your equipment just in case that way you can hopefully just drop the image onto their box and roll.

 

[tom11011]

Not sure you got the full story there.

The server was able to boot up but only to the point where you can't login to it? The raid did its job.

 

[warrend110]

Nerm,

what is the update on this situation? I had a similar issue last week.

dcdiag basically said the domain did not exist.

They just had one large single volume in RAID5. They had backups but only of their data (eg. quickbooks, documents, PMS, etc).

As an update it was decided to rebuild from scratch so I spent 14 hours yesterday rebuilding the server from scratch and then rebuilt their VM of their RDS server of which they did not have backups of either (honestly though being used only for RDS that isn't that big of a deal). Today I get to rejoin computers and migrate users to the new rebuilt domain.

EDIT: They only have 25 workstations but they are spread out over 3 locations so some travel is involved.

 

[Nerm]

Unfortunately I had to wipe and rebuild the domain from scratch.

 

[ohio_grad_06]

Ouch.

 

[Markverhyden]

Bummer......

 

[Nerm]

It wasn't really all that bad, yes lots of time involved of course (but that was all billable so no tears there). Doing it from scratch allowed me to redo and correct some things that had been done by the original tech that I wasn't all that fond of anyway.

 

[ohio_grad_06]

Now create a backup image and keep on hand in case the user interferes again lol.