Server Backup Solutions

[brandonh]

Hello, long time lurker here. I am a solo operation and have very small businesses as my clients. My largest client (~25 users) recently sold to a private equity firm so they are putting the entire network under the microscope. One area I need to address is server backups. I am a bit embarrassed to say that I have been using Windows Server backup > USB HDD enclosure for local backups plus a cloud backup solution (Code42). I would like to upgrade to the local backup system and am looking for recommendations. Money is not an issue but I don't want to completely overkill for the size of business.

Some equipment and services that they are or will be using.

• Server A: Hosts DC, Azure AD and WSUS VMs
• Server B: Applications Server (Quickbooks and a large db SQL Server), Shared Files and client File History backups
• Microsoft 365
  • Exchange
  • OneDrive

I'm looking for advice for a solution and best media for hot swapping to rotate off site. TIA!

 

[Sky-Knight]

Get a Synology, sync it to a cloud provider, enjoy not having another monthly.

 

[HCHTech]

I second this opinion. For SMBs Datto, while very capable is overpriced - I couldn't get any of my clients to bite. Synology bundles Active Backup for Business which you can use to backup your servers as often as hourly if you have the capacity, and Hyperbackup can backup the Synology to A3 or B2. Storage costs are very low.

 

[Sky-Knight]

Not to mention Active Backup for M365 which works shockingly well.

 

[timeshifter]

Has Synology improved their apps and backup products a lot over the last couple of years?

About 4 years ago I set up a Synology NAS for a small business and was not impressed with the backup software they provided that copied user files to the NAS. The products you're mentioning here look really interesting. Does Synology offer any kind of phone support now? That was a negative back then too.

I started offering Solarwinds backup solutions around that time. Works pretty good and reasonably priced. They're now called N-Able. Agree with the Datto pricing, only have one client on it but they're also my largest and most dependent on the application on their server.

 

[Sky-Knight]

@timeshifter Active Backup for Business is Datto, for "free". It's not files, it's system images on a schedule.

Active Backup for M365 uses Microsoft Graph to suckup the entire M365 tenant... there's a GSuite version too.

Support is still a bit of an issue, but I haven't needed it. They do respond, it just takes a day sometimes.

 

[timeshifter]

Very cool. Happened to be working on a Synology DS218+ this week. Expanded from two 4TB drives in a SHR to two 8TB drives. Couldn't have been easier. This client happens to use Google Workplace and was making some changes. I just downloaded and set up Active Backup for Google Workspace. Very cool!

 

[Sky-Knight]

@timeshifter A week from now you'll be where I was back in December... WTF was I doing NOT using this?!?

My only real gripe is the software needs a Synology platform to run, so you cannot restore the stuff without another Synology. If you use HyperBackup to stuff it all in Backblaze, you'll need to source a new Synology, slap HyperBackup on it, connect it to your bucket, and pull down the backup. Then you have to relink the backup apps to their backup data... THEN you can restore. Which can be a rather lengthy process.

So if you do this, make sure you have a plan for WTF to do when Synology dies.

I keep meaning to play with this stuff: https://xpenology.org/

To see if I can turn a crap box into an emergency unit of some kind. haven't gotten around to it yet.

 

[timeshifter]

A week from now you'll be where I was back in December... WTF was I doing NOT using this?!?

Well, my answer would be when I tried some of their backup stuff 5 years ago I wasn't impressed. Either I was doing it wrong or they've really improved in 5 years.

I keep meaning to play with this stuff: https://xpenology.org/

I set up a machine with that a year or two ago. It was a fun project and it might make sense for an in-house experimental deal, but I recall that the licensing was suspect so I wouldn't want to put too much energy in it. But, I might check it out again. I've got several HP Microservers lying around.

 

[timeshifter]

If anyone is thinking about this, here's what they say about it's legitimacy:

Is Xpenology legal?
The water is a bit murky here, but the consensus is that Xpenology is not legal. Why? Xpenology is a bootloader for Synology’s operating system DSM. It runs on a custom Linux version developed by Synology, where the community has edited the code to work on non-Synology devices.

Doing this is illegal, as you are accessing it for free instead of paying for the Synology DSM. If you are using Xpenology for personal use, chances are you won’t get caught, and it should run as normal. But if you are using it for business or anything other than personal-lab projects, we would recommend avoiding using Xpenology.

It might not be as reliable as Synology DSM and could land you in some hot water if you are caught using an illegal system to run your business. Synology has stated that Xpenology is illegal and violates their intellectual property, cautioning you to avoid using Xpenology on another server. Synology warns that you can face legal action using Xpenology.

In some ways, it’s similar to “Hackintosh,” a concept where macOS is run on hardware not produced or assembled by Apple. It’s illegal, but the chances of Apple going after individual users are uncommon. They did, however, shut down an online shop selling pre-built Hackintosh computers. Synology would likely use a similar approach. You are most likely safe to build Xpenology yourself, but buying it off someone could land you both in hot water!

Remember to take what we say with a pinch of salt. After all, we aren’t lawyers and can’t guarantee that Xpenology is 100% safe to run! Make your own judgment call here and avoid purchasing a NAS xpenology from anyone!

 

[Sky-Knight]

@timeshifter I know the legality is problematic, but that's not the point. I'm looking for a tool to afford a space to restore data more quickly in the event of a total Synology failure when I cannot get my hands on a Synology due to supply shortages.

That's a massive problem with the current situation Synology has created for themselves. I've had this conversation with Synology sales and trying to get to their legal team too. So far everyone is quite clear that my efforts are an emergency stop gap and not an attempt to bypass Synology's monetization.

I'd rather this all be FOSS but... can't have it all. I may wind up having to get another Synology to keep online here just to have something to use for a client in a pinch. It's not THAT much money all things considered. But it would be nice to be able to slap together a crap box to be on a white horse. Such things have saved my bacon more than once in my career.

 

[YeOldeStonecat]

First....ask the client "How long can you afford to be down, if your server caught fire and melted into a pile of charcoal"? Or something like that.

Really need to seriously ask that question before proposing any solution. If the client can afford to go days or a week without a functional local network...sure, go with some old school server backup that spits copies to a NAS. And when the server goes "BOOM'...you spend a day looking at the dead server, deciding if/how to fix, or order a replacement...secure the money, oh yeah..."hurry the order"..so you're stuck with a less than ideal base model server instead of a quality proper specc'd one with good RAID and drives. Receive the server, build it, and now we're days later and starting the restore process while crossing our fingers.

Bah...I'm done with that headache stuff.
I only push Datto now. Their Altos are cheap and affordable for the small clients and work great. Siris for clients with heavier needs...best of the best of the best, period.

So nice to not break into a sweat when a client calls and says their server went "boom". Just a nice relaxed feeling that a Datto unit is there...and "OK, we'll have you back up in a jiffy, no problem!". Been doing this "IT for biz clients" for 30 years, no other backup product (and believe me...I've used a LOT)...gives me the relaxed confident feeling that Datto does. I do not want to support anything else, too old and value my time too much to deal with anything else.

Hardware issues are on them, not me. Software issues are on them, not me. Daily tests of the servers, emailed to us....takes that task out of our daily jobs too!

...and why wouldn't any client want their IT guy to use a product that makes them feel that way?

For cloud data...like 365, DropSuite, or Dattos own SaaS backup...we use both.

 

[Sky-Knight]

@YeOldeStonecat At the price of the Sirus, you can afford a Synology that can also act as an emergency hypervisor, and yes it'll fire a VM up made out of a server backup.

Still... Datto BDR retains the crown for offsite replication. If the entire site burns down, they push a button and those systems are online in Datto's DC with OpenVPN ready to connect clients to them. That's a huge deal! And the only place I know of to get it!

The rub? All on premise workloads should be in the cloud by now, and once you've moved it all into the cloud you just don't need Datto anymore. You can Azure Backup the VMs easily enough and very cheaply. But in these cases once again Synology gives me tools Datto just doesn't... because I can cheaply use that VPN tunnel to image the Azure VMs back to the Synology on premise.

Now a business owner that's afraid of the cloud really has no reason to be afraid, because he has a magic box on premise that's taking his cloud based business and pulling it all out of the cloud into a box he can put his hands on. I can do that with a Datto, it just costs too darned much to do it. And $80 / month for 1 year of 2TB retention? They're deluded. I used to have a substantial amount of Datto out there too. But it's 2021 now, and not a single client I have wants it anymore. They've all gone to Synology in the last few months, and they know if their place burns down they're going to have to wait until a new Synology shows up for me to fill it with their offsite junk so they can restore.

But what do they care? They're all single office SMBs, if the office burns down they aren't working for several weeks anyway! So you're right to ask that downtime question. I understand the value of Datto quite well, and I give my customers the choice. But don't discount the Synology on the bottom, you're missing out on a VERY solid solution that's almost as easy as Datto for a fraction the cost. Which means you can sell the "support plan", and make money off it in circumstances the client wouldn't bite on a Datto.

 

[YeOldeStonecat]

For cloud servers....yes, other options can be better than Datto, since it's just as quick to roll back an Azure server up in the cloud versus a physical local/on prem server. Are the OPs servers up in Azure, or local?

We've had a Synology RS2212+ in our office for quite a while, have played with its add-on for 365 backup..."meh". But I'm not going to spend (waste) time rolling up my sleeves to re-invent a mcguyer version of Datto out of duct tape and bubble gum on a Synology. I'll stick to automated, and something supported 100% by its support.

I see "cost too much" thrown around when talking about good backup solutions. A basic Datto Alto...has a "cost" of well under 100 bucks/month for one server, the Alto appliance is FREE, you resell that for well under 200 bucks/month to the client. I don't want to spend time in the "backup your server" space for any less than that....I need it to work, and I want someone else (the vendors support department) to work the screwdriver and wrenches if it has a problem. No way I could offer my prices that low if I'm cobbling together my own version of a Datto via Synology and I'd still never get all the things Datto does (how does the Synology do the daily image bootup/screen verification test, and log it?)

 

[Sky-Knight]

The Alto is limited to 2tb, and that $100 / month for unlimited retention is about 2x what everyone else whats for the same money. Besides, for the clients I have that still have on premise investments 2TB isn't close to enough space, since that means 1tb of protected storage.

In the past, I'd agree with you. But Datto is doing too much, and their support too strained. It's simply not worth the money anymore.

Do you have any idea how many times I've run into a "passing" Datto backup that still doesn't work? How many times I've had a FAILING Datto backup that DOES WORK! That test is a joke in and of itself, well proven to be the worst kind of security... the false kind. I got tired of being on the phone with their support. I haven't had need to call Synology ever, except once... for the presales support call.

I'll sell my clients a Synology at half the price of the Datto, pocket the difference and rely on myself thanks. I'd still be using Storagecraft myself, but Arcserve bought them and now they're in the same boat of degrading performance over time. But at least with SPX I can control where the data is, and know what it's doing. With Datto it's a giant black box that has let me down for days while "support" figured it out.

Datto cost me a $6000 / month contract two years ago! Thanks to the above, thank goodness I managed to get the DC back online, it was only an archival SQL server that was shot for a few days. But again that was ME getting it going, not them. So what did I pay for again exactly?

I would never use them again, but Datto still is the king of off-siting... but that doesn't work as well for me as it does for you either. They're still stuck in PA, and don't have anything running out on the other side of the continent for me. The emergency hosting they provide is barely functional as a result.

 

[YeOldeStonecat]

Heck our invoice with Datto is over 6k/month. But me make twice that back in gravy.
haven't had a restore fail, and we've done a good amount of restores, including offsite spinups. Haven't had any Datto box be down for "days"....on the rare cases we've had an issue with a restore going south. Their support has been great it getting things back up so quick, if we've had to have them intervene. Pointing back to the "comfort, I won't break out in a sweat" thing.

Cost wise...I still don't want to go less than 150 bucks/month. For backup logs to hit our general ops distie group, each and every day, and have someone at our office put eyes on those emails each day, and take action to correct an issue if a failure comes in, and tie into monthly server maintenance, document stuff, etc. Yeah, no iDrive or carbonite or mozy or whatever backup for us. Top stuff, top price, but it takes up our time...so we start our billing higher, thus price doesn't come in. Clients purchase a "service...that we're managing their backups"....not just a product that may, or may not work when the crap hits the fan.

 

[SAFCasper]

If you want a cheap but robust and feature rich backup look at purchasing a refurbished server. Throw in a ton of internal storage and install Veeam Backup & Replication. You can even run the community version completely free for up to 10 VM's with practically 99% of the features available. No support but Veeam has such a huge community it's rare you need it.

Run regular backups to the local storage. Run backup-copy jobs to either rotated USB, NAS, AWS, Azure, or anything S3 compatible like Wasabi or Backblaze.

Some of our smaller single server clients I've actually installed Veeam on a dedicated Dell OptiPlex with Win10 Pro. Spec it with an i5, 32Gb RAM, 256GB SSD for OS and 2TB SSD for backups. Powerful enough to run their VM's temporarily should the production server fail.

 

[Sky-Knight]

Heck our invoice with Datto is over 6k/month. But me make twice that back in gravy.
haven't had a restore fail, and we've done a good amount of restores, including offsite spinups. Haven't had any Datto box be down for "days"....on the rare cases we've had an issue with a restore going south. Their support has been great it getting things back up so quick, if we've had to have them intervene. Pointing back to the "comfort, I won't break out in a sweat" thing.

Cost wise...I still don't want to go less than 150 bucks/month. For backup logs to hit our general ops distie group, each and every day, and have someone at our office put eyes on those emails each day, and take action to correct an issue if a failure comes in, and tie into monthly server maintenance, document stuff, etc. Yeah, no iDrive or carbonite or mozy or whatever backup for us. Top stuff, top price, but it takes up our time...so we start our billing higher, thus price doesn't come in. Clients purchase a "service...that we're managing their backups"....not just a product that may, or may not work when the crap hits the fan.

Oh I agree, I'm not less than $200 / month myself. It takes two hours of time each month to monitor whatever I'm using for backups, doesn't matter if it's Datto or something else. If the client has the service hours to cover that, it's not a line item. But if they don't... it is.

That was the mess with Datto is after they cost me that huge client right as the panemic hit I started realizing that I was spending all the same time keeping them going as I was doing Shadow Protect on my own. The difference being that I was making less money for that time, and the client was paying more. So I bailed. My Datto rep really tried too, he's a great guy. And the support agents were awesome too, I was running into bugs in the platform they couldn't fix themselves, which is where the three day mess came from. You've worked with me long enough to know how good I am at finding edge cases!

Synology is a relatively recent addition to the mix for me, and I've been quite happy with the way it works. Especially the Active Backup for M365 relative to Datto SaaS backup. I've had less trouble, and the interface is simple enough my business owners are in some cases reading emails of their employees right out of the admin panel.

Of course, none of the above is possible if the client doesn't have a full managed network stack at the office so I can put the Synology on its own VLAN along with an Untangle or equivalent to control access. Clients also have to sign a waver warning them of restoration delays due to the limits of their Internet connectivity. If they want rapid cloud restores, they're going to have to pay for Drop Suite, or Datto SaaS.

@SAFCasper If Veeam ran on something other than Windows, I'd be using it. The problem is by the time you're done with the Windows license and the cost of that refurb platform, you're out the cash for an extremely well equipped Synology. It IS a powerful option though! I've also had some very good conversations with Veeam on this topic. They're aware of the limitations of their platform, and the day they jump to Linux is the day I abandon everything else because I can then have Veeam in a K8S cluster running here AND int he cloud at the same time covering my entire client base. That's the ULTIMATE solution! But it cannot be done at present.

 

[SAFCasper]

Yeah Veeam running on Linux would be great I'm with you there. Would also feel better from security standpoint. But on the downside I imagine you would loose all the great REFS features since Microsoft keep that closed-source. One of our setups has 48TB of backups on a 20TB volume due to all the REFS witchcraft it achieves.

Pricing on a server for Veeam vs Synology depends on your scale. Smaller setups mostly the Synology is going to win because as you say, Windows Server licencing is a big hit. You can use W10 Pro for small setups but not ideal.

When you need more horsepower Synology starts to get expensive. Like in this OP example where we have 5 VM's including a large SQL server. If you want the Synology to run those VM's at a useable performance level then it's going to need Xeons and a ton of RAM. Minimum something like a RS1619xs+ and bump it up to 64GB RAM which comes in not much short of £2,500 for me. And the CPU still isn't that great at just a quad-core 2.2GHz (35W TDP). A refurb server in that price range would obliterate in performance.

 

[Sky-Knight]

ZFS and BTRFS both have similar featuresets to ReFS. They would just have to pick one and get good at it. They both have deduplication, which is how you're getting the apparent savings you're seeing with ReFS.

For my use case, I'm mostly using the Synology for M365 backup, not even using it to backup servers / desktops. The clients either don't have servers at all thanks to cloud everything OR the ones they do have are in Azure and using Azure backup. In these cases the Synology is only using the image backup so that we have something OUTSIDE of Azure for extreme emergencies.

It also means the client owns all their own stuff, and they aren't dependent on me for anything.

But goodness if we could get that into K8S? So many things would change.... so much magic would be possible. Microsoft is getting into this space too, so perhaps we can use Windows containers on Server 2022? I haven't had a chance to test it yet.