Keeping log files small

TAPtech

Well-Known Member
Reaction score
521
Location
Fairfield County, CT
Hi guys,

I'm sure you've run into this scenario before on MS server 2008:

Space on the C: goes to zero, you run a program like WinDirStat (thanks StoneCat for recommending that) and see that you have a 20+GB log file from SQL.

I have been setting recovery models to simple for these, in most cases it is the sharepoint log file which none of my clients use.

Does anyone know how to set a limit to the size this log file can be? I'm using SQL Management Studio Express.
 
To stop the log files from growing you mean?

No need. And, as far as I'm aware, you shouldn't. You just create a maintenance plan to back them up regularly. The log files will automatically shrink following a successful backup.

This is roughly the TSQL I generally use to perform the job:

Code:
DECLARE @BackupFileName varchar(100)
DECLARE @DateTimeStamp varchar(100)

SET @DateTimeStamp = REPLACE(REPLACE(REPLACE(CONVERT(varchar,GETDATE(), 13),'-','-'),':','-'),' ','_')

SET @DateTimeStamp = LEFT(@DateTimeStamp, LEN(@DateTimeStamp) - 4)

SET @BackupFileName = 'FULL_PATH_TO_FILE_NAME' + @DateTimeStamp + '.bak'

BACKUP LOG DATABASE_NAME TO DISK = @BackupFileName WITH CHECKSUM, STOP_ON_ERROR;

RESTORE VERIFYONLY FROM DISK = @BackupFileName


GO

(replacing FULL_PATH_TO_FILE_NAME and DATABASE_NAME of course)

I configure the TSQL script to run each day (followed by the usual maintenance clean-up task to delete backups older than 'x' days).



That's roughly the gist of it ... if you need any more help/details let me know.
 
Yes, to stop them from growing. Regular backups do make more sense though. Will your script overwrite old files? I don't want an issue with the backup folder growing indefinitely too :)
 
No. Like I said, I would usually just add a maintenance plan to do that. You can do it with a TSQL script (or even an external batch script) if you prefer, but a maintenance cleanup task makes setting dates/retention far simpler and easier to manage.

Just add it to your backup plans. This is one I set up for a customer:

4I1akXm.jpg


ihlRs9v.jpg
 
Actually, in answer to your question: If you remove the date-stamp and save the log with the same file name every time, then yes it will simply delete/overwrite old files (no maintenance necessary), but I prefer to retain log files for 'x' days just in case a specific roll-back point is required, in the event of a backup-recovery.



EDIT: No problem, you're welcome. If you need any more details/help, just ask. :)
 
This! I do this on SBS all the time, else C drive bloats.

Really? But, backing up the transaction logs prevents drive bloat too and, as far as I'm aware, is considered best practice and generally 'the proper way to do it'. In many ways, I would say it's more important to backup the logs regularly than the DB itself, after all it's the logs that give you the transaction-by-transaction changes to the DB.

Having said that, I suppose it does depend on how important the database is though. Most of the DBs I work with are for POS systems, where every database transaction is important. Regardless, it's easy enough to implement log backups, so it's something I'd recommend doing no matter what the DB contains.

I think transaction logs are one of the most misunderstood, and yet important, features of SQL databases. They're really not that mysterious though. This article explains their function very well:
http://technet.microsoft.com/en-us/magazine/2009.02.logging.aspx

Crash recovery is only possible if the transaction log is intact. In fact, the transaction log is the most important part of the database—it's the only place where all changes to the database are guaranteed to be described in the event of a crash.
If the transaction log is missing or damaged after a crash, then crash recovery cannot complete, leading to a suspect database. In that case, the database must be restored from backups or recovered using less desirable options, such as emergency mode repair.
 
Last edited:
Having said that, I suppose it does depend on how important the database is though. Most of the DBs I work with are for POS systems, where every database transaction is important. Regardless, it's easy enough to implement log backups, so it's something I'd recommend doing no matter what the DB contains.

And there's the "A-hah" moment where a lightbulb should go off in your head.

SBS comes with a ton of crap by default. Honestly nobody....nobody..cares about WSUS logs. And Sharepoint logging. And various other SBS maint routines. WSUS? Who ever has to refer to those? You can wipe that clean to zero on a weekly basis and not miss one bit of important info. Not to mention most biz clients are on an RMM package for updates anyways. Sharepoint? What percentage of SBS users really actually use sharepoint? 10% maybe? MMmmm...even that is generous, I'd say <5%.

I've been doing tons of SBS installs since the early days of Back Office 4.5 and SBS2000...it's been my bread and butter...and esp with SBS2K3 and onward, the management of those default useless SQL logs has been the bain of most SBSrs...just neutering that logging saves tons of time, and has never...ever ended up in a "whoops..I shouldn't have done that" moment.

Now...important SQL databases, you mention POS (for point of sale I assume)...or other line of business software...obviously you do want to ensure that it's backed up properly and you don't neuter those maintenance routines. But of course we'd transition this thread in another direction..and the first point of that discussion is of course installing, and locating, those databases properly...on another partition, not the %system% drive.
 
I wanted to chime in earlier here but figured I would let the big guys leave the good information in the post before I ruined it :D

I completely agree with all points in the conversation thus far.

Moltuae- on important databases, proper backup is not only very important, but also can help with this space issue.

StoneCat and freedom- setting recovery to simple for the built-in "this many GB's wasted for that, really MS? Really?" is the way to go. In fact that's what I've been doing most of the time.

I have a few clients with E2 shop system that run the SQL version, and they have full recovery model with backups to keep space appropriate.
 
And there's the "A-hah" moment where a lightbulb should go off in your head.

SBS comes with a ton of crap by default. Honestly nobody....nobody..cares about WSUS logs. And Sharepoint logging. And various other SBS maint routines. WSUS? Who ever has to refer to those? You can wipe that clean to zero on a weekly basis and not miss one bit of important info. Not to mention most biz clients are on an RMM package for updates anyways. Sharepoint? What percentage of SBS users really actually use sharepoint? 10% maybe? MMmmm...even that is generous, I'd say <5%.

I really hate to have to say this (and believe me I honestly do, because I usually agree 100% with everything you say and I totally respect your vast knowledge of all things server), but I have to disagree here.

I think you may have fallen into the common trap of mistaking SQL transaction logs as .... well, just that, mere 'logs'. I made the same mistake myself for a while when I first started working with databases. The examples you give, WSUS and Sharepoint logs (unless I have totally misunderstood the purpose of both of those, which is entirely possible) are essentially diagnostic logs.

Now I'm not claiming to be an SQL expert, nor do I even know enough to call myself a DBA -- I just know enough to get by to be honest and, like everyone else, I'm learning all the time -- but I think I do understand a little about SQL transaction logs, and diagnostic logs they're not.

I think one of the main reasons transaction logs are so commonly disregarded and misunderstood is their name; 'logs' is such a misleading term in this case. Yes, they log the transactions (ie the DB changes) but they're so much more than just logs. Their functional relationship with the database probably more closely resembles that of the VSS snapshots of NTFS. They're not some passive log that should be ignored or discarded, they're an active record of database transactions that not only provide the detailed log of events of each and every database change, but a means to revert the DB back to any point in time with single-transaction resolution.

Referring back to the link I posted earlier -- I'd highly recommend reading the article in its entirety for a full understanding, but to reiterate the with the snippet I posted previously:
Crash recovery is only possible if the transaction log is intact. In fact, the transaction log is the most important part of the database—it's the only place where all changes to the database are guaranteed to be described in the event of a crash.
If the transaction log is missing or damaged after a crash, then crash recovery cannot complete, leading to a suspect database. In that case, the database must be restored from backups or recovered using less desirable options, such as emergency mode repair.

I've been doing tons of SBS installs since the early days of Back Office 4.5 and SBS2000...it's been my bread and butter...and esp with SBS2K3 and onward, the management of those default useless SQL logs has been the bain of most SBSrs...just neutering that logging saves tons of time, and has never...ever ended up in a "whoops..I shouldn't have done that" moment.

I don't doubt your experience and knowledge of servers for a minute -- it's far greater than mine, but there are tons of examples of configurations and procedures that, while considered to be bad practice, may be employed for years without any significantly dire consequences, but that doesn't make them the right thing to do. A couple of common examples: Running a server as part of a workgroup instead of as a domain controller, or even failing to maintain regular backups; both are common scenarios that you might 'get away with' for years, but I'm certain that, like me, you wouldn't recommend them.

Sure the risks of major issues in this case are minimal -- the more likely scenario is that you may introduce a number of minor database corruptions over time. Those corruptions may be so minor that they cause no issues and go completely unnoticed, but isn't it better to eliminate the risks all together, especially considering the highly granular database recovery ability that properly maintained transaction logs also provide?

Setting up regular transaction log backups is not only 'best practice' but as easy to do as setting up regular database backups. It's also, arguably, more important than regular database backups AND (here's the best bit) it solves the growing log problem, elegantly and properly. Why compromise?
 
Moltuae- on important databases, proper backup is not only very important, but also can help with this space issue.

Absolutely. Backing up transaction logs is also best practice and simple to do. Simple recovery mode just seems to be a bodge that saves very little time -- I see no advantage to using it apart from the time saved. And, to be honest, it probably took me longer to write the posts in this one thread than it would've taken to configure 5 or more typical transaction log backup schedules. It's easy, worthwhile and billable - a no-brainer as far as I see it.
 
I think you may have fallen into the common trap of mistaking SQL transaction logs as .... well, just that, mere 'logs'. I made the same mistake myself for a while when I first started working with databases. The examples you give, WSUS and Sharepoint logs (unless I have totally misunderstood the purpose of both of those, which is entirely possible) are essentially diagnostic logs.

At no point did i say that SQL TLogs are just logs and i don't think Stonecat is that stupid either. The point we were both trying to make is that for SQL databases like WSUS and SharePoint (if not used) then the ability to roll back the database to a particular point in time isn't really needed. By setting the recovery mode to Simple i can still recover back to the last backup and worst case i might of lost a days worth of WSUS data...no great loss. This saves the admin overhead of individually backing up the TLogs for those databases and monitoring the jobs.

For a LOB SQL database i would always 100% recommend using a FULL SQL recovery model and backing up the Tlogs.
 
At no point did i say that SQL TLogs are just logs and i don't think Stonecat is that stupid either.
And at no point did I say you said that, though I think you're being just a tad unfair calling people who misunderstand transaction logs stupid. It's a very common misconception to assume they're relatively unimportant. I've worked with many intelligent server techs who didn't fully understand them. I'm sure there are plenty of things I don't understand that you do, and vice versa.

The point we were both trying to make is that for SQL databases like WSUS and SharePoint (if not used) then the ability to roll back the database to a particular point in time isn't really needed.

And I agreed with your comment; yes, if it's it not an important database it's not really necessary. So I think we're actually in agreement, even if it doesn't appear that way! lol :)


And my point is simply that (assuming the database is important) why use simple recovery mode when it takes so little time to configure the preferred method of transaction log backups to manage their size?
 
And I agreed with your comment; yes, if it's it not an important database it's not really necessary. So I think we're actually in agreement, even if it doesn't appear that way! lol :)


And my point is simply that (assuming the database is important) why use simple recovery mode when it takes so little time to configure the preferred method of transaction log backups to manage their size?

That was exactly my point that you misunderstood and jumped on the soap box against. I had a hunch the OP was talking more about SBS. And after a few hundred SBS installs and years supporting them...I'm quite comfortable in knowing what can be killed on them...without a single dire consequence.

As I also stated earlier...important databases, line of biz software, etc...of course, don't skimp on those. But the ones that crunch the C drive on default SBS installs, which I believe the OP was talking about....I won't lose a minutes sleep mis-treating those and nearly shutting them down. After all....greater majority of those are simply not used. They're just default unwanted "bloat" on an SBS install...and they cause problems. Take 15 seconds or so to "simple recovery mode" them..and the problem is gone for the rest of the life of that server.
 
Ah. Ok, sorry, I did misunderstand you in that case. And yes, I'd fully agree with that.

In my defence, the OP did say "MS server 2008", which I didn't take to mean SBS. But to be fair, looking back, I see you did mention SBS early on.

Tell you what, let's just blame the OP. ;)




To be totally honest, I've never installed an SBS. I've always been put off by their limitations; they don't sound like a very scalable solution.
 
Ah. Ok, sorry, I did misunderstand you in that case. And yes, I'd fully agree with that.

In my defence, the OP did say "MS server 2008", which I didn't take to mean SBS. But to be fair, looking back, I see you did mention SBS early on.

Tell you what, let's just blame the OP. ;)




To be totally honest, I've never installed an SBS. I've always been put off by their limitations; they don't sound like a very scalable solution.


I humbly apologize from the greatest depths of my soul, and kindly ask for mercy.
 
Back
Top