New Dell BIOS updates cause laptops and desktops not to boot

[Porthos]

New Dell BIOS updates cause laptops and desktops not to boot

Recently released Dell BIOS updates are reportedly causing serious boot problems on multiple laptops and desktop models.

www.bleepingcomputer.com

 

[sapphirescales]

This is why automatic BIOS updates via Windows Update are a TERRIBLE idea! I've seen my fair share of bricked laptops thanks to this.

 

[Sky-Knight]

And yet, thanks to the intelligence baked into the hardware, we're rapidly approaching the need for monthly BIOS updates.

So buckle up, this is going to get worse, before it gets better.

 

[britechguy]

I can't recall the last time I've had a BIOS update "go south."

I don't expect any uptick, nor do I think we'll ever reach a day when one never, ever goes wrong, either.

 

[sapphirescales]

And yet, thanks to the intelligence baked into the hardware, we're rapidly approaching the need for monthly BIOS updates.

So buckle up, this is going to get worse, before it gets better.

Thankfully a lot of systems have a built in backup BIOS in case things go wrong. I don't understand exactly how these work or what would happen if the system tried to install the bad BIOS updated 10x times through Windows Update. I know that the ROM has limited write cycles, so would it eventually kill the main BIOS ROM? There's got to be some sort of procedure for when the wrong BIOS update gets applied, it fails, and it rolls back from the backup BIOS. Maybe something that tells Windows Update not to try to apply that update again? How does Microsoft and the OEM know these BIOS updates are even failing? Do they track how many updates fail so they know when every time they try to install BIOS update #928498383 it fails on every machine so they know there's something wrong with the update? I can see this getting very problematic in the future if it's not set up properly.

 

[GTP]

The only time I ever had an issue with BIOS update was when I bricked a Traxdata 2X DVD burner trying to get it to 4X.
It was my fault.
It seemed to be taking forever so I killed the process. Unfortunately it killed the burner too.

I was kinda peeved at my hastiness considering burners at that time were $600 +

 

[Diggs]

I don't understand exactly how these work

I remember the old Gigabyte MBs had two BIOS chips on board side by side.

 

[britechguy]

Do they track how many updates fail so they know when every time they try to install BIOS update #928498383 it fails on every machine so they know there's something wrong with the update?

You're talking two different issues here. There could be UEFI/BIOS install failures, which would be issue one, and UEFI/BIOS updates that cause machine malfunctions, which are issue two.

Telemetry as it exists today could track issue one. Issue two is thornier depending on whether the malfunction(s) involved cripple any ability for telemetry to send data. If that's the case, it will be pretty much as it always has been: logging support calls and tracing the reason(s) for them back to a specific update.

 

[Sky-Knight]

Thankfully a lot of systems have a built in backup BIOS in case things go wrong. I don't understand exactly how these work or what would happen if the system tried to install the bad BIOS updated 10x times through Windows Update. I know that the ROM has limited write cycles, so would it eventually kill the main BIOS ROM? There's got to be some sort of procedure for when the wrong BIOS update gets applied, it fails, and it rolls back from the backup BIOS. Maybe something that tells Windows Update not to try to apply that update again? How does Microsoft and the OEM know these BIOS updates are even failing? Do they track how many updates fail so they know when every time they try to install BIOS update #928498383 it fails on every machine so they know there's something wrong with the update? I can see this getting very problematic in the future if it's not set up properly.

From what I've seen there are two predominate fail-over models for BIOS.

The first is by far the simplest, which works like most of our SOC systems. If you've ever worked with a 3D printer, or large machine lathe they're pretty similar. Cellular phones work this way too! The mainboard has a primitive boot loader that runs first, usually checking a specific USB port for a storage device, and if present looks for a specifically named file. If all of the above is correct, it doesn't even bother to engage the BIOS and instead loads the BIOS from the USB stick.

Assuming all that works, and POST completes, there's a 2nd process that's either manual or automatic that reprograms the BIOS on the mainboard with the above successfully POSTing BIOS.

The 2nd, more automatic recovery utilizes two BIOS chips, often built into the same assembly so physically there's 1 chip, but it's electrically split in half. In the event that an update faults the first one, the second is used to boot and once the system is online it flashes the 2nd chip's contents over the first. If the 1st chip is successfully updated sometime after boot completes the mainboard will copy the 1st over the 2nd.

Either process works, but the latter is better for remote support for obvious reasons.

 

[sapphirescales]

The 2nd, more automatic recovery utilizes two BIOS chips, often built into the same assembly so physically there's 1 chip, but it's electrically split in half. In the event that an update faults the first one, the second is used to boot and once the system is online it flashes the 2nd chip's contents over the first. If the 1st chip is successfully updated sometime after boot completes the mainboard will copy the 1st over the 2nd.

I was only aware of this type of BIOS recovery. What I'm concerned about is what if Microsoft keeps trying to install this bad BIOS update over and over again? Don't most BIOS chips have a very limited number of write cycles? Unless Microsoft can tell that the BIOS update has failed and doesn't try to do it again, I don't see how this won't become a problem.

 

[NJW]

Don't most BIOS chips have a very limited number of write cycles?

Not a practical limit, I imagine. A Winbond W25Q16BV (random selection, about 10-year-old design) data sheet quotes 'More than 100,000 erase/write cycles'.

Even allowing for manufacturing optimisation over the years, I can't see that number reducing to the point where lifetime is a concern. 4,000 cycles gives more than 10 years of daily reflashes. You're more likely to hit a power glitch before the chip wears out.

 

[Sky-Knight]

@sapphirescales A legitimate concern, but not functionally real in quite some time. The chips used are comparable to USB sticks, and even the cheap memory modules in those can handle tens of thousands of rewrites before it's a problem. At most I've seen Windows update do a BIOS update once a reboot, so I cannot imagine actually wearing the BIOS out during the service life of the mainboard in question.

I'm sure I could kill one given enough time and a script to force the issue, but that's not happening here.

 

[sapphirescales]

@Sky-Knight My knowledge about BIOS chips is very out of date then. I guess it makes sense they had such limited write cycles with flash memory being so expensive in the 90's, and I guess it makes sense the quality has gotten better nowadays seeing as flash drives are so cheap. Also, I'm sure modern BIOS chips need to store a LOT more data than they used to with UEFI, mouse support, etc. They basically run their own little OS! Though it's worth mentioning that even in the 90's there were motherboards that had mouse support in the BIOS. It wasn't common, but it was possible even with those tiny, low capacity BIOS chips.

 

[Sky-Knight]

@sapphirescales Yep, like everything else the game has changed several times in that space, and for the most part up until recently we didn't have to think about it much.

But sadly today we're in a world where we have that mini-OS, and it's vulnerable to all sorts of nasty things so you have to patch it.

And this is why I use Dell almost exclusively, and to a lesser extend Lenovo. Their BIOS update processes are safe and easy to perform remotely. And I must be able to do so to maintain the fleet.

 

[Galdorf]

had his happen to customers dell laptop she is a bookkeeper that is quite computer savvy it did the updates asked for restart when it restarted it waited an hour dead screen tried powering it off and on nothing brought it to me i removed battery and cmos battery and hard drive for few hours cycled power button put them back tried powering not even a slight screen flash dead.
Tested power supply on identical model with no battery it powered up fine looks like it bricked it good.

Took back off tested power on motherboard had the proper voltages i have a background in electronics as well so it is not a broken power jack battery was also fully charged took meter reading off it.