Sorry, got mega screwed on appointments today, I'll try to respond to all below with context...
@YeOldeStonecat
Don't use the Sharepoint admin center....use Teams!
OK I'll look into seeing if teams can help me get a better idea of what amount of data is being stored in what folder. Do you have any tips on getting this information? Is there some sort of WinDirStat functionality I can use for the Teams Channel?
Let's peel back several layers here.....
*How is this organized? It's not one big bucket, is it? Sharepoint has matured over the years, increased "max file count" as well as greatly increased "max file size".
HOWEVER...you still want to manage things...
**Create a few separate Teams. Teams that...not everyone is a member of. Not every has to see the office/admin folders. Not everyone has to see HR folders. Not everyone has to see sales folders. Not everyone has to see...<insert more>
I see what you are saying here, basically their old file server that they retired had a Z drive, which was basically their DATA DRIVE on their on prem server which contained ... EVERYTHING. The Z drive was migrated into their primary Team General Channel Files, so there is essentially 20 years of Data that was in that on prem server and now that data exists in their General Channel which is roughly 1.8TB now.
They somehow went from 1.3TB after 20 years, to adding almost an entire 1TB of data in one year. They have probably tripled the size of their company in one year due to the massive amount of rain we have received in Southern California with them being a roofing company. I'll need to look into retention to see if it's possibly set too high and saving too many versions.
This is also unexpected because they apparently "stopped" storing images for jobs in their server since they now use other cloud based management systems specific to their industry to facilitate each job and those cloud services handle their pictures now. Or so they say.
In the old days of on-prem servers, we (many of us) used to create a whole separate mapped drive...per major department. A common "S" drive..shared...for everyone. An O drive for office. A Q drive for Quickbooks/accounting, a P drive for personal, an M drive for ....you get the point. Same thing with Teams!!!
Now, that "30 - 60 minutes to sync".....is every user syncing EVERYTHING? Refer to....is this "one big bucket"? Or...are you breaking up into individual teams, also with additional channels under each time? And...only those who need to access it, sync it? You're doing files on demand? Or..keeping everything local?
As stated above, basically their whole company is in that General Channel. When syncing the general channel yes, everything is left as files on demand, but it has to sync something like 300k changes everytime we sync the channel into OneDrive.
What kinds of files are they? Any designers there running some design software like archiCAD? A lot of designers?
All of their data is basic Word, Excel, PDF, and Pictures. I set them up with a higher end on-prem precision workstation running a RAID1 with 2x250GB NVme SSD's strictly to host their QB data file. That's it's only job, host Quickbooks.
What data can be archived? For a lot of my clients, I have an "Archive" team . I had them put "old stuff there". Stuff they rarely, if ever, have to go scroll through to dig up something. Yet it's there for them. And..never a reason to "sync" this...just access it through the Teams app. (and I retire outgoing users OD files there too).
I have no doubt that data can be shuffled around to different Team channels as I said above, their retired on prem Servers Z drive is now in their default Teams General Channel. This STILL does not help with the amount of data they are storing.
For clients that have staff that create TONS of Teams....I don't have everyone sync every Team/channel. Just what they need. Adding too many file syncs to OD will bog down older machines, heck...there's a point where it will to potent new rigs too. So...I have the talk with staff..."Us older dinosaurs that have been working with mapped drives and File Explorer since the days of Win3x...yeah, I find they like syncing it to File Explorer. But new kids..they "get" cloud apps, they're good with navigation of OD and SP through the "File Open...or File Save" options of Office apps. And...they "get" working through Teams to get to the files.
The old "in the days of on prem servers...you just added storage". Well, that cost money. It was often after hours work too...so that labor fee was high. And don't forget...that increased backup costs too....of those old on prem servers. And you have that "gotta replace the server every 5 years..or maybe 7 if some people stretch things....and the price of that new server, with lotsa new hard drives....just keeps going up...and up".
Understood. The type of data they store can be backed up by a simple file backup solution like Crashplan. It has worked well for me in the past for clients that store non-database type data. Granted I understand recovery will take longer but you can't beat Crashplans costs to backup simple data like this. Crashplan has come to the rescue MANY MANY times for me in the past.
So...a couple of twenty dollar bills a month for 365 storage suddenly isn't really that much more expensive. Yeah...it costs money to hoard files! And it costs money to backup tons of hoarded files. Their lack of housekeeping isn't a problem you have to solve...it's a conversation you should have with them though, and you can certainly make suggestions to them to groom things. If they want to keep things manageable and affordable. Turning to some 3rd party service to store it at....to save a buck or three a month...for the big increase if overhead in managing that data, (aka more time you spend)...and..oh, still should back up that data via 3rd party too....
I understand housekeeping is not my problem, but shuffling data around won't fix the storage issue. It's not just a couple of $20 bills a month. It's $20 per 100GB of storage, so to get them another 1TB of storage will be another $200 per month on top of the already $200 a month they are paying to bump them from the default 1.3TB they have from their licensing to the 2.3TB it is now. So essentially $400 per month for 3.3TB of storage.
When moving a client from on prem to 365...I always have AT LEAST one sit down meeting with them ahead of time, to "go over" what's on that file server. And I lay out what their goal should be. It's been my experience that...when looking at on prem file servers, say the business has been around for ..oh, 30 years. Might have had the same IT guy, or had several IT guys. But often server migrations just end up by doing basically a "copy/paste" of old servers junk...to new server storage. Repeat in another 5-7 years for the next server replacement. Repeat again. And again. Staff come and go, yet folders remain, nobody really "takes ownership of that big pile of folders". RARELY...does someone take the time to say "Hey, now is the time to really look at everything, and...you get to take some brand new shiny file cabinets and re-organize all this stuff just how you want it! Wipe the slate clean and reorganize how you want it!"
Agreed, not my job.
And you can make suggestions on organizing, but the actual organizing...is not your job. I also point out that I will make an archive Team for them...but for now I am making an archive folder on the old server that they can move "all their old junk" to...which I'll put in the Archive team.
Still doesn't alleviate the AMOUNT of data.
@callthatgirl
I have them buy a 2nd account, cheap and then they can view it all on all devices. Worth the cost. Be sure to add backup to that account, the nightmares I can feel coming lol.
Please elaborate? I'm not talking about OneDrive/Mailbox data, I'm talking about SharePoint tenant data. Each tenant comes with a default storage amount of 1TB per tenant + .15GB per licensed mailbox. Are you suggesting buying a whole new tenant with a whole new domain?
@Sky-Knight
Sharepoint versions, so you aren't talking about 1 copy of a file you're talking about N copies of a file based on retention policy. Have you defined retention? If not... well start there because you need to do that.
This I need to look into. Retention policies for SharePoint have been left at default, and I'm not sure even what they are. I do also have their data backed up via DropSuite as well.
As Stonecat points out, if they use the storage they need to pay for the storage. They can do so via capital investment, or ongoing expense. So are you really solving a problem being worried about $20? It's a cost of doing business, and the cost of manually digging through those libraries and killing old stuff is real too.
It's not $20. It's $200 per month for another 1TB of storage on top of their already $200 per month for an addition 1TB on top of the default 1TB + .15GB per mailbox license.
So yeah, this is file server creep, were you ever the guy that had to delete stuff on the file shares? The answer is no... so why is it your job here? If management doesn't want to pay for the extra storage, they can take the time to delete stuff to clear room. Either way, they're spending money. And I'll bet once you put things into that context they'll continue to just pay more for hoarding, because it's CHEAPER than paying an employee to spent billable time deleting stuff.
I will present them to this as on option. Delete data or pay more. If I did want to offload that data into "cold" storage, what service would you recommend I push the data into? B2?
Supposedly Archiving is coming to SP in 2023. Supposed to be cheaper storage. I think I stumbled on a post somehwere when looking up a similar question - looking for a method to systematically move only certain files from SP to another provider like Azure files.
Otherwise find a cold cloud storage solution and migrate old untouched data off of SharePoint into cold storage.
@putz
Supposedly Archiving is coming to SP in 2023. Supposed to be cheaper storage. I think I stumbled on a post somehwere when looking up a similar question - looking for a method to systematically move only certain files from SP to another provider like Azure files.
Otherwise find a cold cloud storage solution and migrate old untouched data off of SharePoint into cold storage.
This seems like it should already be an option similar to mailbox archiving. Would you suggest maybe a B2 bucket as a "cold" storage option?
@SAFCasper
If the data is such a mess you can't differentiate current VS archived data... then the client needs to organise their data. Sometimes clients need told the hard truth - there is no magic technical fix to years of neglect and poor organisation. Someone needs to sit down and manually organise that data in a more more manageable fashion. Anything else is just pushing the issue down the road, where it will likely snowball into a much larger issue over time.
Agreed
Maybe do some analysis to estimate how much data can be archived (eg. files not modified in 5+ years) then explain this in real business costs. For example reducing their SharePoint usage by 500GB would save the company $6,000 over the next 5 years. They may suddenly be motivated to get organised when that realisation hits.
Understood. Until the person who is delegated this job botches it
