O365 Service Issues - ongoing

[HCHTech]

Head's up, everyone:

status.office365.com reports:

Title: Can't access Microsoft 365 services
User Impact: Users may be unable to access multiple Microsoft 365 services.

More info: Users would be unable to access Outlook.com, Microsoft Teams including Teams Live Events, and Office.com. Additionally, Power Platform and Dynamics365 properties are affected by this incident. Existing customer sessions are not impacted and any user who is logged in to an existing session would be able to continue their sessions

Current status: We've identified that reverting the recent change did not alleviate impact to Microsoft services as expected. We're working to explore additional options for mitigation.

Scope of impact: Any user may experience access problems for Microsoft 365 services.

 

[Sky-Knight]

Yep... someone broke it!

But to be clear, this is far more than M365... Azure's Directory Services are offline.

So if you have an Azure integrated service... it's busted.

This INCLUDES laptops and desktops that are using Microsoft.com logins!

 

[HCHTech]

Move to the cloud - they said...
Look at all of the advantages, they said!

I'm being facetious, of course, but the whole "single point of failure" thing is an undoubtedly undervalued downside of the whole "Everything M365" movement.

 

[Sky-Knight]

It's been my gripe too...

There is however a massive catch. Most SMBs will not spend the money to properly mitigate the MULTIPLE points of failure present in an infrastructure that's dependent on a cheap NAS, or overgrown desktop pretending to be a server.

Yeah, MS screwed up today. But this is the first hard down in the platform's history of this scale. That's a ton less downtime than lack of UPS fried the power supplies and or hard disks. And we haven't even gotten past basic environmental issues here for potential problems.

 

[thecomputerguy]

Love getting 20-30 panic calls that take 3 hours of repeated redundant explanations of unbillable time.

 

[Sky-Knight]

That's what an auto-attendant greeting is for...

But yeah.

 

[Paul Rodgers]

Looks like it’s resolved. Authentication lag that was resolved by rerouting traffic. My day ends at 6 generally and I kept going until things cleared up just to let clients know what was going on and that it should be resolved tonight and definitely by tomorrow morning.

 

[Markverhyden]

but the whole "single point of failure" thing is an undoubtedly undervalued downside of the whole "Everything M365" movement.

I always mention this when I'm talking cloud vs self hosted.

 

[Sky-Knight]

It was mostly fixed before I clocked out at 11pm AZ time, now it's 7am and all systems seem green.

My tenant access was cut for about 3 hours, during which I needed Teams and Planner to get some stuff done which was highly annoying but I still got it all done. On top of texting people like mad via my phone... which was more annoying because I'm far too used to using "Your Phone" on my desktop to bounce off "Your Phone Companion" on my Pixel 3a to SMS directly from my desktop through my cell phone. Which of course requires a Microsoft login so that was actually the first thing that died for me, and one of the last to come back.

Someday I'll learn how to swype as quickly as I type... but it is definitely NOT this day!

 

[HCHTech]

Everything seems back to normal today. At least here in EDT, the problem didn't start until after official work hours and was fixed by the beginning the next day's work hours - could have been a lot worse. I'll bet there are a whole lot of new meetings scheduled for MS folks involved in this...

 

[YeOldeStonecat]

Seemed to be less than 2 hours of duration for our clients. From around 5pm to a hair after 7pm or maybe 730 (I'm EST)

Soon as I got home yesterday, my wife pounced on me with the "Email errors on my phone". She has an iphone...kept going in the enter password loop, and then the broken graphic icon of the login.portalblah blah. So I figured it was just her phone...as I got home about the time the issue started..and went right to the kitchen to get dinner going. So I'm blaming her phone..."you won't use Outlook, this here issue on your phone is one of the many reasons we don't support the crappy iOS email client. Probably had some update that broke its ties with the MS Auth App". And kept going down that rabbit hole about that..and maybe her MS Auth App didnt' get updated. Or...or...something else iphone related.

I soon cracked open my laptop, fired up a browser, went to office.com...went to log in as her to disable her MFA for troubleshooting...and BOOM!. Went to log in as me..same thing...BOOM".

Minute later went to Tonys 365 FB Group...saw the post that is just started.

Ah well had to eat crow for my bad mouthing my wifes iPhone to her last night...LOL.

Sent out a bunch of tests to gold/silver clients and really didn't get much at all.

Zero issues this morning.

 

[britechguy]

"These things do happen occasionally," is my standard response to outages. And that's 100% true and there is absolutely nothing an end user can do to fix it. You just wait until the outage is resolved.