Large amount of Storage/Backup solution

[ohio_grad_06]

As some of you know, I work at a helpdesk. The organization I work for has about 6-8 different divisions that effectively act as independent companies. However, everything network related resides in our IT Department.

A bit of a backstory, we inherited 2 servers from our Publishing Division, they were essentially at one point running their own psuedo IT Department. Kind of against policy, but they had the room in the back of the building and guys who could work with things enough to be dangerous, and bought a couple of servers, and were running those, and ended up a year or so ago being forced to move them to the actual IT Department.

Fast forward until today. When we got these servers, there was no backup being run on them. They have one box that is running Windows Server 2003 R2, and has a data drive of about 5TB. The other system is an HP proliant of some type of other, older(we usually stick with Dell, but they purchased this without our knowledge at the time). But the 2nd system is running Windows Server 2008, the data drive is a size of 5.35 TB.

Effectively at this point, we are using probably about 4.5TB of that space. The older server has a couple of TB left, but from what they are telling me they are going to have to add more data. For backups, until we could get something else, we got a couple of large, external hard drives, and use those to try to get a full backup every other week or so. Not ideal by any means.

Their manager talked to me today, they'd like to move to something a bit more robust, which is a great idea. The environment here is that IT has a domain set up that the whole building uses. Their deparment specifically has a smattering of Macs and PC's. I'd say about 10-15 Macs, and probably close to that number, maybe more PC's mostly with Windows 7. The Mac guys do a lot of design work, like articles, graphics etc. They produce a lot of books, tracts, magazines, etc.

According to the manager, he thinks they've got 8-12 TB, he knows enough to be dangerous on IT/Network stuff. My initial thought that I proposed to my IT director is maybe consider a couple of large NAS devices. We have some older Buffalo equipment that we use for other divisions that is set up, they work off one NAS device, which replicates to a backup each night, and puts things they delete into a trashbox folder in their user folders, which works well enough for what they do. So that thought has crossed my mind, but the Buffalo devices seem to have trouble sometimes with domain permissions etc.

We'd like though, for the 2 servers I mentioned earlier to get them onto some type of robust, enterprise grade Server/NAS and backup solution, but I'm still learning more about this area. Wanting to see what you guys all think. I've heard guys toss around the name Synology, but don't know a lot about it. What recommendations would you all make here? We do have a corporate dropbox account, but that seems like too much information to upload to the cloud.

Also, we do have most of our equipment rack mounted, so if we can get something that would be rack mounted, that would be a plus. Forgot to mention, the servers are running RAID 5, they are interested in something that does incremental backups as well for the occasional deleted file. I know the Buffalo NAS devices in the past like we use had a trashbox feature which moved a file into your trashbox folder if you deleted a file, but the idea was you could go into trashbox and pull it out again.

 

[BlackLabTechs]

I think you have some questions that need to be answered before deciding on a backup solution. If one of those servers goes down, how long can they be down until it's restored? How many versions / points in time does the backup system need to keep? How much data can they lose? (Backup hourly, daily, once a week?) If you need / want cloud, how much of that mutli-TB data changes frequently? If very little, once the initial seeding to the cloud is done (can even be done by shipping a drive out) then the delta's will be manageable depending on internet pipe.

I've had great success with Synology but depending on your answers above, you may find a different solution that would work better.

 

[ohio_grad_06]

Right now, as I said, the backups basically consist of I'm having back up to a large external drive attached to each server(each server has it's own). The backup software for one of them took a week to run recently, which is way too much, So I basically end up backing up their data, using cobian backup at the moment and have it compress it to a 7zip file. I know there is a better solution to it. At the time it was better than nothing. Then I can usually use the previous version history to go back to older files. It looks like using the previous versions function, I can now go back about a month, but a couple of weeks may be sufficient. I don't usually have many requests for recovering a file, maybe once every few months or so.

As far as how long they can be down, they would basically want it back yesterday. In other words if it goes down, they'll want it back asap, because all of their design work etc is on those, and they are mapped to the shared folders there, and work directly off of that. How much multi-TB changes, I would say probably not a lot, because once they get editions of books, magazines etc done, my guess is they shelve them and move to the next project, but occasionally revisit.

Internet pipe right now is a 100mb fiber connection from Windstream. Organization total, probably 125-150 work in the building on a regular basis. Their division I'd say 30-40 people.

 

[BlackLabTechs]

Sounds like the server(s) are just file servers then w/o any "real" services on them. I would toy around with the idea of backing up to a NAS (rackmount available) and then if something happens to their server you can always share out via the NAS w/o having to wait for server recovery. Have the NAS backup to the cloud with mutli revisions. In the event of total destruction and the need to pull back down that much data from the cloud - appears most of the files could wait until processed as they are older and not immediately needed.

A week for a backup to complete? Something's gotta give there.

 

[ohio_grad_06]

I don't know of any particular services really, just mainly file servers. I'd honestly be ok with just bumping them over onto a NAS and decomissioning the servers personally. One is Server 2003 R2, so that's getting long in the tooth, and the other is Server 2008, which honestly that's going on 8 years old too.

Any recommendations on a particular brand or NAS? My boss is not really hip on the Buffalo units. They work, but sometimes getting them to play nicely with the domain etc leaves a bit to be desired. Any particular backup software or anything that you like? I'm honestly still learning when it comes to the more corporate solutions in that area.

When I first started here, I was more of a break/fix tech. I've learned a lot sure, but much of the equipment was already here when I came, so I'm kind of learning/wanting to learn more about the enterprise grade solutions others are using and recommending vs the home grown stuff.

 

[Knightsman]

We setup a Synology RS2416+ as our NAS for our backup system. I dont think you need quite as big, but similar models have less bays.

 

[SAFCasper]

With two Synology units you can setup high availability. One NAS is assigned as active and the other as passive. Data is constantly replicated between them and should the active NAS go offline, the passive NAS will kick in and take over.

Info and setup guide here - https://originwww.synology.com/en-g..._availability_configuration_with_Synology_NAS


NOTE:
Just like RAID - this is not to be considered a backup solution. It won't protect you from deleted files or corrupted data and you don't have any versioning.

Not sure how I'd back this up as most of my clients are <1TB so it's much larger than I'm familiar with. I will leave that for someone else but do consider restore times. Even over gigabit LAN I'd be surprised at anything less than 45 hours to transfer 12TB of data.

 

[putz]

IMO you need to determine what data is active data and what data can be archived and does not need to be apart of the daily backup plan. I have run into many customers who have terabytes of data that they think they need to backup on a daily basis but when you drill down into it, they really only have maybe a TB of data that is ever active/hot data, if that. The rest are just old files and are never modified, just used once and a while to go back to for reference.

maybe create some partitions on your file server where you have one partition that current, relevant data is stored and shared, and another partition that holds the archived or cold data. you could make it so people only have read only to the archived data and if they are constantly requesting certain files to be modified or changed, move that data to your active data partition. This way your daily backups can include the active data partition while excluding the archived data. That allows you to set longer backup retention on your active data. Then you can set another backup for your archived data, maybe back it up once every few months and your retention could be less for those backups.

Otherwise if you are really trying to backup 10-15TB of data as part of your routine backup set, its not going to be cheap, especially if you need to be able to have many recovery points available or if the rate of change on the data is very large, not to mention initial backup and restore times for any given backup set is going to be outrageous. Certainly having management or the owners define their RTO and RPO would give you some idea of what type of hardware and software you need to match that and how your backups should run. The last thing you would want to do is implement a solution that cant meet their expectations.

I wouldn't recommend using NAS for primary storage devices in a larger multi-user environment, especially Buffalo units, they are horribly slow and they require special formatted drives that they charge and arm and a leg for. You are also making it more difficult to control file/folder access to your end users and audit that data if needed. Not to mention if you have the data on the NAS but are running backup software on another server then its going to be extremely slow to backup because your storage fabric is essentially a gigabit copper link. They are fine for offsite storage for disaster recovery purposes but I wouldnt recommend them for day to day file storage for a larger multi-user network.

Make sure you understand the difference between backup and redundancy because they are not the same thing. Data redundancy will not allow someone to go back 3 months from now and recover a file at a specific time of day so having a trashbox on your NAS or syncing one NAS to another will not give you a true data backup system.

 

[marley1]

Didn't read whole thing. Synology on site with the storage needed. Use something like Veeam or StorageCraft to backup the servers. Another Synology Offsite

 

[ohio_grad_06]

Thanks for all the input, things have been really busy here. Working on setting up a more up to date terminal server, setting up new wifi in our building using Cisco Meraki, kind of nice if you've not used it, cloud managed wifi. Downside is I think there is a yearly fee, but you configure everything in the cloud, I've read of some guys sending one to a remote location, and having it configured before it even arrived, and someone just had to plug into the network.

Before I get on a tanget though, I did some quick looking here's what I'm wondering to start with. The buffalo units look tempting, but we do run active directory and have had trouble with the older ones at least attaching to the domain at times. I found a netgear unit that looks nice. Anyone have any input on this unit? Looking at the 16 tb unit with the idea of running raid 5, which would leave us about 12 tb. Then maybe if I can consolidate the backups as suggested, having them archive some older stuff, so that it could be on the server, copy on the nas, but in a seperate area, and maybe a final copy on their existing external drive.

Here's a link to the netgear unit.

https://www.cdw.com/shop/products/N...N31843E-NAS-server-12-TB/3921155.aspx?pfm=srh

Also, software wise, I was thinking about using aomei backupper in a technician license. The cost is reasonable, and from when I've used their products in the past, they seemed to work well enough. Also, keep in mind that we have about 30 servers in our data center so it's not a standard small business setup, looked at Veeam, haven't really gotten to use it yet, but at roughly a grand per box, that could get high quickly. Any thoughts?

http://www.backup-utility.com/technician.html

Thinking this as a start, then I could check and see if my boss wants to look more into offsite storage etc.

 

[fencepost]

RAID5 on modern high-capacity drives is groveling for problems in the form of "Wups, a second drive failed during the rebuild, sorry about your data," and in a unit with no capacity to hold a hot spare either? I'd consider using it for someone's home collection of ripped movies, but not for anything more important than that.

And frankly for the $2500-3000 that they're looking at for those if you're willing to cheap it out with a 4-disk RAID5 I'd rather buy a used server that holds 6-8 3.5" drives and a good SATA-compatible RAID controller that'll support large drives. Do a 5-drive RAID6 plus a hot spare, or maybe do a 6-drive RAID6. It's not ideal, but neither is that 4-bay Netgear you're looking at regardless of whether it's 1U.

 

[SAFCasper]

According to the manager, he thinks they've got 8-12 TB...

Looking at the 16 tb unit with the idea of running raid 5, which would leave us about 12 tb.

So they potentially have 12TB of data right now and you are looking to purchase a NAS with 12TB of usable storage... using all 4 available bays.

HUGE alarm bells going off right there. What if their data grows? You have no room for expansion.


PS.
An alternative suggestion at a similar price - Drobo B810i. 4x 4TB drives + 240GB SSD Cache.

http://www.drobo.com/storage-products/b810i/

 

[ohio_grad_06]

So what I'm hearing is the other NAS would work, but not be ideal.

I haven't presented anything to them yet, but would something like this then be a better idea?

https://www.cdw.com/shop/products/N...x-10GbE-RN422X64E-100NES/3098855.aspx?pfm=srh

With that drobo unit, you say it's 4 drives but that 4 drives aren't preferable?

If this were a client you all were dealing with, what would you try to sell? Obviously most things here will be better than what they have, but I don't know what they will want to do.

 

[fencepost]

Looking back up through the thread, I think it may make sense to step back and re-list exactly what you're looking for because I think I see (at least) two different things but we're only talking about one.

First, they're on old servers with issues that you need to get them off of. They think they have 10TB+ of data "live" on those boxes. So you may potentially need a solution for "live" use with a lot of storage - much of it almost certainly old/archive data as @putz pointed out. Even if it's not frequently accessed "live" data is probably going to need to be handled differently from backups, e.g. SAS instead of SATA.

Second, you need backup for that system, with versioning, and able to grow with the output of a graphics department cranking out new content at a fairly solid clip. There might also be some desire to "seed" earlier files into that from the current backup system such as it is but that's probably more trouble than it's worth - just hang onto those current drives for a while and those are the "pre-current-system backups." The backup is going to need to hold everything the live stuff has plus multiple versions, and how much space the versions take up is going to depend a lot on the backup method (e.g. full changed files or just diffs?). This can probably go onto SATA drives in a big array. For these drives I'd actually look at the information Backblaze provides on the drives that they work with, though I think that building your own storage box on their platform may be overkill.

 

[BlackLabTechs]

If you decide to go with a NAS, look for an expandable unit. For example, in my shop I have a Synology DS1515+. 5 bays for today but easily expandable to 15 bays when required and still controlled / accessed through the main unit.

 

[SAFCasper]

With that drobo unit, you say it's 4 drives but that 4 drives aren't preferable?

There is nothing wrong with using 4 drives. If my post came across that way it certainly wasn't intentional.

The point I was trying to make is with the ReadyNas you aren't leaving much space for growth.

You say they have 8-12TB of data now - what if that increases to 13TB in 10 months time? Your NAS is now too small and you find yourself buying four new 5TB or 6TB drives to increase it. Quite an expensive upgrade and also time consuming as you need to create a new array then restore all the data.

By all means go with the ReadyNas and 4 drives. They are good units. But personally if my client had 8-12TB of data I would wan't at least 4x 5TB drives -- but probably go for 4x 6TB just to be safe and "future-proof"



As for the Drobo - yes, it has 4x 4TB drives but there are 8-bays. With Drobo you can add disks "on the fly" to increase the capacity of an array. So if you need more storage in the future just add another 4TB drive into one of the empty bays. That's it - upgrade done.

 

[ohio_grad_06]

Thanks for the input. I'll keep everyone informed, but we had a consultant come in yesterday working on our SUN system that we used to use with Oracle and a program called Pledgemaker. Anyway, my boss started to ask him about what enterprise grade equipment they sold, and he mentioned we already had it as part of the SUN system, but just may need a newer copy of backup exec, as we are on an older version. So going to look at that path and see what happens. Drobo sounds interesting though if you can start small and toss drives in on the fly.