Solarwinds N-Central remote client domain admin credentials theft

[fencepost]

Quick summary since I closed my tabs for it:

Huntresslabs confirmed then announced (after 90 days) that it's possible for anyone who can download an agent from your N-Central server download the agent/probe configuration file which may contain domain admin credentials for your clients in plain text.

To clear the credentials you can either go into each client & location, then Administration, Defaults, Agent & Probe Settings, Credentials tab and clear/change values there (while noting which ones have them or don't), or you can do the bulk setting and go to the same place in your SO (top level), set junk and check the Propagate box which will push it down to everything.

Propagating just the password might be the simplest thing to do if you have thousands of endpoints and hundreds of clients/locations (looking at you @YeOldeStonecat ), though I haven't tested it.

My approach was to turn on the firewall rules I put in place back during the Apache Struts thing so only client sites with static IPs are able to reach our N-Central server, then go through and clear the 5 clients that actually had credentials set.

Edit to call out folks who've talked about Solarwinds, just in case: @TAPtech @Slaters Kustum Machines @Frick @HCHTech (but I think you're on RMM) @nextechinc @freedomit @marley1 @Rosco

 

[Sky-Knight]

Kaseya, Automate, and Continuum all do this too... if you can get an agent installer, that installer has the credentials buried in it in clear text.

The thing is, that agent should be locked behind an MFA protected authentication portal. If someone is at that level, they already own you.

 

[fencepost]

The N-Central agent runs as Local System, I think this is more a factor of the probe that can be used to push agents onto domain-connected PCs and needing the credentials for that.

 

[Sky-Knight]

The N-Central agent runs as Local System, I think this is more a factor of the probe that can be used to push agents onto domain-connected PCs and needing the credentials for that.

Sure, but every RMM does the same. Which is why you pull the credentials after the initial push. I thought this was standard practice? Though it would be nice if the RMM people would at least encrypt the things. They all make some stupidly bone headed decisions sometimes.

 

[Mike McCall]

Sure, but every RMM does the same. Which is why you pull the credentials after the initial push. I thought this was standard practice? Though it would be nice if the RMM people would at least encrypt the things. They all make some stupidly bone headed decisions sometimes.

I find it interesting how some RMMs still are 32-bit agents and therefore unable to support OS X Catalina. Too many of them seem to have their Dev's focusing on eye candy and features rather than functional substance. I've used a lot of them and have seen it on every one of them.

 

[Sky-Knight]

@Mike McCall Agreed, they are focused on what sells to make money, not what makes good tech. That sort of cancer is industry wide though...

 

[YeOldeStonecat]

Interesting..and not good.
Credentials are baked into the probe..and the site specific agents...domain\username and password.
I do know that a recent update we did to our on-prem N-Central dealt with tightening up security as far as communications 'tween the agent and our N-Central server....all of that got encrypted via TLS or something.
But to have creds in plain text...to be found when cracking open the bundled up agent/probe installers...ugh.
I'm so slammed and behind with work right now ...trying to think of how to handle this...ugh.

 

[YeOldeStonecat]

The thing is, that agent should be locked behind an MFA protected authentication portal. If someone is at that level, they already own you.

While the login for the RMM is behind MFA....sadly...the http links to deploy (allow download) of the agents are NOT behind MFA....I know this because I can copy the link for a clients agent..and send him the link in an email, and they can download it.
So...I'm sure somehow someone can run a long sniff against a hosted RMM...and eventually come up with the links of all of the downloads available on it. Probes and agents for every_single_client on that rig.

 

[Sky-Knight]

While the login for the RMM is behind MFA....sadly...the http links to deploy (allow download) of the agents are NOT behind MFA....I know this because I can copy the link for a clients agent..and send him the link in an email, and they can download it.
So...I'm sure somehow someone can run a long sniff against a hosted RMM...and eventually come up with the links of all of the downloads available on it. Probes and agents for every_single_client on that rig.

Oh goody, so we either ACL the deployment mechanism, or pull credentials out of the RMM entirely... great.

 

[fencepost]

It's not even really much of a long sniff. They start at 100 I believe, and each client/client site gets that number incremented.

Some of this may be going away on the N-Central side with the separation of agent communications from regular 443 access. That may make it feasible to simply lock away 443.

 

[gpg]

Very interesting that this issue came up. I am in the process of trying to find a secure RMM. Was leaning towards Kaseya and BeyondTrust. Have been using ConnectWise Control but may have to ditch them. Most likely going with BeyondTrust at this point.