Exchange Server 2010 customers sometimes ask why their server disk drive is filling up with log files. Usually they are referring to the transaction log files created by the mailbox databases.
Update February 2013 – there is a specific issue with iOS 6.1 that causes excessive log generation on Exchange servers. Click here for more details.
Each Exchange 2010 mailbox database comprises two main parts:
- the transaction log files
- the database file itself
The folder containing the log files will look something like this.
A best practice for Exchange 2010 mailbox servers is to store the database and transaction log files on completely separate disks. This is to protect the server from data loss if one disk or the other has a failure.
The way this works is that each database change is written to a memory buffer and also recorded in a transaction log file. Periodically the memory buffer information is also written to the database file. When this occurs a checkpoint is updated that tells the server which transaction log entries have and have not been written to the database yet.
If the server was to unexpectedly restart, the database comes online in a “dirty shutdown” state and the checkpoint is used to tell the server which transaction log entries need to be replayed into the database to recover the information that was lost in the memory buffer when the server failed.
Over time these transaction logs will grow, because of course the mailbox database is continually changing as new mail arrives in mailboxes (as just one example). Eventually the log files will fill up the disk if they are not removed.
To remove the transaction log files the database needs to be backed up. When an Exchange Server database is backed up by a proper application-aware backup product, after the backup is finished the backup program will issue a command to VSS (Volume Shadow-copy Service) on the server that the backup was successful and to go ahead and truncate the transaction logs.
The server then proceeds to remove the transaction log files up to the nearest checkpoint prior to the backup commencing. Because the database can continue to change and write new transaction log files while a backup is in progress it is not unusual for multiple transaction log files to still be present after a backup has completed. However most of them will be removed, and regular backups are the method by which transaction logs can be kept from consuming all free disk space on the server (as well as the obvious benefit of having your Exchange databases safely backed up).
So if your Exchange Server disk is being filled up by transaction log files, the issue is likely to be one of the following:
Cause: You aren’t backing up the mailbox server
Solution: Back up the mailbox server with a proper Exchange Server application-aware backup product. There are commercial products available for this such as Symantec Backup Exec or you can use the built-in Windows Server Backup for the task.
Cause: You’re using the wrong type of backup
Solution: Make sure you’re running a backup job type that will truncate the logs. Full and Incremental backups will truncate the transaction log files, whereas Differential and Copy will not.
Cause: The backup is not completing successfully
Solution: Check your backup product for log file entries that indicate what the issue is.
Cause: The backup is completing successfully but transaction logs are not truncating
Solution: Check the Application Event Log on the mailbox server for errors with the log truncation process.
I have Exchange 2010 and running windows server backup software for VSS Backup. It is now more than 24 hrs and logs showing that “Running consistency check for application exchange”. My drive storage is draining due to these logs. How to overcome if running consistency fails ?
Hi I had an issue where my free space was disappearing in front of my eyes
Enabled Administrator account & saw where the issue was
Exchange log files
Trouble is 20 Gig in one day ,my question is why
Did the exchange service Pack 3 upgrade will see how this goes
Hey Paul. Thank you for all the information I have gleaned from you in the past. I would like to comment on the ‘log files filling up the drive’ problem that has made recent news. I found 1 mailbox server (exch 2010 sp3) having this problem; the backup would disconnect after 3 minutes. I hade to re-register vsswriter. I am in the midst of another problem (again 1 out of the 4 mailbox servers): backup completes but log files still accumulate. In addition System Event Log displays a 7011 every 30 seconds. I think it may be the SCEP Client. I will know by the end of the day. Anyway, I was trolling while I waited on this backup to complete and thought I would see what sort of problems you had going on…Thanks,
hello Paul … need your help !!!
i have exchange 2010 in my office .. the drive where my database is stored its full now and in exchange management console my database is showing unmounted. how can i increase the drive size do i have to connect another hard drive and expand the size of the drive or i can simply delete some of log files from that drive to get some space. because all the users are not receiving any mails and some of users can’t access their mailboxes through outlook. its showing disconnected.
The Real Person!
Author Paul Cunningham acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
You should expand the volume so that the database can be mounted.
If the logs are accumulating because you’re not running backups, then you should immediately run a backup once you get the database mounted.
Hi,
I have 1 data base that logs always increase.
I have already make backup but maybe backup not cleaning the logs and i have to delete manual.
Can you help how to solve this case ?
Hi Paul,
I have the issue on an new Exchange 2010 that after moving a couple of mailboxes from an Exchange 2003 that the Transaction Logs folder grew like 50GB in 2 or 3 hours. Currently 45 users with total mailbox size of 40GB! My guess is that I should monitor the use with ExMon to find the cause. What would you suggest? Thanks
The Real Person!
Author Paul Cunningham acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
The migrations themselves will generate a lot of logging. Could be as simple as that. But yes, check whether one of those moved mailboxes is causing the excessive logging.
Thank you Paul , Wish you a Very Happy New Year.
I tried using Windows server backup to backup the exchange database on to NAS drive and it was stuck with “Running a consistency check for the application Exchange” for 18 hours and eventually I had to cancel the process.
The database size is 195 GB. What could be the cause?Any Suggestions.
Hi Paul,
I have exchange 2010 and mailboxes are backed up daily using backup assist. The Exchange server & AD reside on Virtual machine. The exchange drive is now left with 10 GB space and I wanted to clear few logs. Noticed that the public folder database has got 72 GB of data and the logs are stored from 2010. The backed up mailboxes (.pst) are maintained for for 2 years and then backup assist deletes the mailboxes.
I transferred few 2010 logs on to a network drive to check if I get any error but and had no issues.
Is it safe to do so or should I dismount the database and then transfer the logs. What issues might occur if I transfer the logs while the database is in mounted state.
Waiting for your reply.
The Real Person!
Author Paul Cunningham acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
Unless you know exactly what you are doing then manually deleting transaction log files is dangerous.
In your situation it sounds to me like your backups are not proper application-aware backups, so the log files are not being truncated. That also suggests that you do not have backups of Exchange that you could actually recover from, which exposes you to significant data loss if you have a database or server problem.
You should really look into that further as a top priority.
Paul,
More a question of curiosity than a “problem” at the moment… Any idea why log files would be accumulating fairly rapidly for a database that holds no mailboxes?
I recently evacuated a database, because it had some underlying damage, but I left the empty DB mounted for a while. get-mailbox -database DBNAME returns no results, and the Exchange Management UI shows no mailboxes in that DB when I filter the list. But logs were still accumulating (187 logs over about 11 hours).
I would have thought that an empty DB had no transactions to commit, but apparently that’s not the case. I’d just like to make sure I didn’t miss something important before I wipe the “empty” DB from the face of the earth.
Thoughts?
The Real Person!
Author Paul Cunningham acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
Might still have a system mailbox, arbitration mailbox, archive mailbox… or it might be that whatever health issue caused you to evacuate is also causing log generation.
If the database is problematic and you’ve emptied it of mailboxes I wouldn’t waste time trying to work out why, just remove it and create a new one.
we have 2 mailbox servers mbs01 & mbs02, with four mailbox database.
mbs01 server
1 in D drive
2 in E drive
3 in F drive
4 in G drive
Transaction logs in T drive
same names & drive letters in mbs02 server.
in mbs01 1 &2 mailbox database is active, 3 &4 in healthy.
in mbs02 3&4 mailbox database is active , 1&2 in healthy.
now we are planning to format our mailbox database partitions D,E,F,G & T (T for Transaction logs) in mbs01 & mbs02 servers, this partition was formatted before with file allocation table 1024 K. now we fill format with file allocation table 64 K.
i have some confusion regarding formatting Transaction logs (Drive letter T:)
i planning to start with mbs02 server first. how can i move my active mailbox database to mbs01 servers.
waiting for your reply.
Hi,
What happen if i try to manual delete the log files?
Thanks. I had no idea.
Exactly what I needed. Thanks a lot !!
Hello Paul, Love the site.
We have a 4 node dag across 2 sites. I have 28 databases and backup works normally most time. (Symantec BE from Active Server only). Sometimes, however, the log files are not truncated. It took a couple of scary log build ups but we made the connection. Log Truncation Issues = Check all DAG Replication!
We’ve experienced the issue of log files building up even with successful backups. This was due to DAG replication issues (occasionally that happens). If ALL the passive database copy have not yet been fully replicated to all dag members, the logs on the Active server are not truncated. Once replication is repaired and Copy queue length returns to 0, log truncation should occur.
This makes sense if you think about it…
Marcina
Hello
Having a similar issue where daily incremental VSS aware backup’s are running ok but the log files are filling up are MBX servers but, on just 2 of our passive MBX servers. While the active MBX log files are larger than normal compared to the passives but still not as bloated. You stated the issue is replication, I ran all replication diag testing and all passed ok for each MBX.
One thing I do see in the logs that there is occasionally communication lost during off hours with our site where one of passive MBX server is at and as result falls out of the DAG. The issue has been identified due to link saturation to the site. Communication dose come backup fairly quick after it goes down and the MBX rejoins the DAG and mail replication picks back up where it left off, and since this a 15 minute blip at the most its not like there are hours or days or mailbox that needs to replicate to the remote site.
Can you share what you did to troubleshoot your issue with mail replication?
Thanks
Hello,
The fact that your passive node log files are smaller than the active one indicates a replication problem.
Did you run the “get-mailboxdatabasecopystatus **” command to see if the database copies are all healthy? If the replication is suspended, the logs will NOT be flushed.
To Troublshoot a particular database, run:
get-mailboxdatabasecopystatus DatabaseName*
This will reveal the state of all copies of that database.
All passive database copies have to be healthy in order for logs to flush.
Think about it. If they are not being replicated, the active node will hold onto the logs until the database can replicate all changes to passive notes.
Marcina
Hi Marcina,
How about if only 1 database that have failed status, other database were healthy.
are other database circular logging would be failed ? or only affected the database that have failed replication ?
Cheers 🙂
Thanks! That’s what I exactly did.
Hi Paul,
I’m running Exchange 2013 and I only have a few users on that server. So currently I don’t have any backup agent installed… Symantec recently just came out with an agent that I plan on installing this weekend. My drive is almost full. What do you think I should do in order to free some space until I can install the Symantec Agent. How should I delete the logs manually to free up some space?
Joey
The Real Person!
Author Paul Cunningham acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
Use Windows Server Backup to back it up.
Pingback: Exchange 2010 FAQ: Why is My Disk Filling Up with Log Files? | petergraphics
Hello Paul, i would like to first thank for your website.
I have a doub that i’ve been discussing with a coworker here.
After a full backup was finished (we use TSM), how Exchange knows that is able to start with log truncation? I mean, where he retrieves the information that the backup was made?
thanks!
Guido.
The Real Person!
Author Paul Cunningham acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
Backup products interact with Exchange via the APIs and so that Exchange knows when the backup is complete.
7 mins to answer!! Thank you so much!
Yes. I am backing up the Exchange 2010 server with the WSB on Server 2008 R2
Event 2006 Cat ShadowCopy
Information Store (6620) Shadow copy instance 7 completed successfully.
For more information, click http://www.microsoft.com/contentredirect.asp.
Event 9780 Event Exchange VSS Writer
Exchange VSS Writer (instance 715f979e-b037-4ecf-a78c-edae2f7f0206:7) has successfully completed the full or incremental backup of database ‘Mailbox Database XXXXX1290’.
The database engine has also successfully executed log file truncation procedures for this database. (Note that this may or may not have resulted in the actual truncation of log files, depending on whether any log files existed that were candidates for truncation.)
Event 225 Task Cat ShadowCopy
Information Store (6620) Mailbox Database XXXXXX1290: No log files can be truncated.
For more information, click http://www.microsoft.com/contentredirect.asp.
Event 11000 Task Cat (11)
Microsoft Forefront Protection VSS Writer has successfully collected the metadata document in preparation for backup or restore.
My log files aren’t clearing and the Backpressure has, from time to time, stopped incoming mail flow. I have changed the setting in the .config file to continue receiving email.
I have the maildstore on a 150 Gig drive. I am using about 29 Gig for the database and log files. Why would this be happening with so much space available.
Thanks for your assistance.
Sam
Silly question, but are you backing up your Exchange? I had a simliar problem above. I was backing up my Exchange VM with Veeam, but I failed to set the job to Prune the Transaction Logs therefore my logs continued to grow very rapidly. As soon as I reconfigured my backups properly, the next back it successfully truncated my log and cleared a few hundred gigs of space. Has been happy ever since…
How exactly did you configure your application to prune the logs. I need your help on this please.
The Real Person!
Author Paul Cunningham acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
You should refer to your backup product documentation for the steps to configure it correctly for Exchange backups.
The Real Person!
Author Paul Cunningham acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
Hi Sam, back pressure may indicate some other issue, as it is not always caused by lack of disk space.
You’ve said your database and logs only use 29Gb together? But how much actual free space is on the disk? Are there other things also using up space?
If it is indeed back pressure you’re seeing, you should look into those event log entries to determine the actual cause of it. Here’s some reading if you need some info to get started:
https://www.practical365.com/exchange-transport-server-back-pressure
Pingback: Exchange 2010 FAQ: Why is My Disk Filling Up with Log Files? « Iordanis Kazanas
I have a question related to my drive filling up, in which I am trying to determine the cause.
Firstly, my backup of the 2010 Exchange Server was completing successfully, but the logs were not being truncated (which is fine as I am ware of this now)
Secondly, my new Exchange 2010 DB Transaction Logs have started to go berserk ! Logs are being created at a rate of about 5 every minute at 1Mb in size each.
I have done alot of searching this morning since I noticed the drive filling up and have tried a few troubleshooting methods. I have run ExMon to try and determine if there is a specific user causing the problem, but at the moment I am stumped.
I have also stopped the ActiveSync app pool just incase it was our iPhones causing a problem, but the logs continued to be created as this very rapid rate.
I was wondering if anyone has come across this before, or can point me in the right direction. Most of the articles I have read, people have used ExMon to identify troublesome users or mailboxes which stand out, but I cannot see anything out of the ordinary using this tool.
Any help would be much appreciated !
I am backing up a exchange 2010 with DPM 2012. How long after a back up does the truncation take place?
was wondering the same… any idea? how long after backup will exchange need to delete those logs?
Hi,
What is the mechanism of choosing hard disk space for Exchange 2010 log files? Any percentage against Mailbox size? (Eg. 1% from DB size or..)
The Real Person!
Author Paul Cunningham acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
It depends on the rate of change within each database. More changes mean more logging. It also depends on frequency of backups.
I tend to go for log volume size of no less than 30% the size of the database volume. Sometimes more if its a high email traffic environment.
More is better than not enough.
Pingback: EXCH - Ex2010 DAG Backup - MCSEboard.de MCSE Forum
Hello,
If any one can answer my query…
My case scenario- 2 DAG Members, 2 HUB/CAS NLB. 5 DB’s in total, out of 5 databases, only 1 of the database on one of the passive node where the database is not mounted log file drive show more utilized space.
If I see the T logs size inside the drive the folder size is only 300+ mb but if I right click to the drive to check the drive size it shows utilized space 45 GB !!!
Where as on the active node where this DB is mounted, the log file drive shows the actual utilized space (300 mb+)
The Real Person!
Author Paul Cunningham acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
Something else must be consuming that space, perhaps a system or hidden folder? Is the drive dedicated to the transaction logs only, or does it also have the database and content index files on it?
Also check that the disk has not been configured as a shadow copy destination for other drives on the server.
Hello Paul,
I figured out the system volume information is the culprit, this is consuming most of the disk space, could you please tell me the way to eradicate this?.
Thank you
The Real Person!
Author Paul Cunningham acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
Did you already check for shadow copies?
I have the same issue with the System Volume Information file using 50 some GB of space. I do not have Shadow Copies enabled, though when I view this I see: F: (Our EDB Drive) Disabled 0 39673 MB on F:
Is there a way to clear this up, and is there a way to free up the space that the System Volume Information folder is consuming?
Thanks,
Robert
I found this to clear up the Shadow Copies
Run command prompt/Admin
diskshadow
list shadows all
delete shadows all
Took care of my System Volume Information issue.
Hi Please can you help me? I have an Exchange 2010 sp1 server on Windows Server r2. The backs up are configured correctly but the Transactiion log just wont clear. the logs are now at 125 and 175 on two mailstores.
Please help
Hi Paul,
We had the issue described above on Friday of last week where by the Ex2010 log files filled our entire log file volume. We are not using an application in the application layer to backup Exchange but instead are using backup direct from storage layer level snapshots, so our VMware infrastructure that Exchange sits on is completely unaware of the backup process. In this kind of environment is there anyway to signal to Exchange that our storage level backup has completed?
Daniel
The Real Person!
Author Paul Cunningham acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
Some storage-level backup products are still VSS-aware and will work. Check your documentation to see if there is an option that is not set correctly.
Otherwise, if you find that your backup product is not compatible, either switch to one that is or turn on circular logging and live with the risks.
In my case I was having problems backing up the server due to a very high amount of log files having been made before I had opportunity to implement backup (65,000) I was down to 3 GB of disk space and this was impacting delivery of mail. In my scenario we have Mailstore Server archiving all incoming and outgoing email on a hourly basis from an Exchange Journal Mailbox.
To remove the transactional files and commit them to the database freeing up disk space I Enabled Circular Logging, this very quickly commits those logs and free’s up your disk space. I would only advise doing this though if you have another way of backing up or archiving your email.
The Real Person!
Author Paul Cunningham acts as a real person and passed all tests against spambots. Anti-Spam by CleanTalk.
Hi Sean, I’ve had to use that method myself to save the day a few times in the past. Definitely has its risks and I agree that a good backup as soon after is highly recommended.