Eight Reasons why your Transaction Log Files keep growing

Introduction

A while back I wrote a short article about the Transaction Log Reuse Wait. Because it turned out to be very popular, I decided to follow up on that article and give a little more detail about the reasons why your database log files keep taking up more and more space.

SQL Server itself actually tells us what is going on with the log files in the log_reuse_wait_desc column of the sys.databases catalog view. So, if you have a particular database that has a log growth problem, you can just run this query to find out more:

However, this query just returns a keyword. To understand the cause of the different values of the log_reuse_wait_desc, we need to dig a little deeper.

Log Records

SQL Server uses the transaction log to guarantee the ACID properties, particularly the durability requirement. For every transaction, enough information to redo as well as undo that transaction it is written to the log file. Before any transaction can commit, SQL Server waits to get a confirmation from the hard drive that the log record was written successfully.

The transaction log gives SQL Server enough information to redo the operations that made up the transaction, should the actual change not make it into the data files, for example because of a crash. Because SQL Server has this information secured in the log file, it does not need to wait for the changes themselves being written to the data files and can commit the transaction while the data page changes still reside only in memory.

Every once in a while SQL Server will execute a checkpoint operation. During this operation data pages that were altered by previous or current transactions are written back to disk.

As a side note, it is possible for data pages containing changes of open transaction to be written to disk during a checkpoint. However, SQL Server has enough information in the transaction log to undo those changes, should the need arise. To have this flexibility SQL Server stores both the redo and the undo information in the log.

Log Reuse

Once every change to any data page that was executed by a single transaction was successfully saved to disk, the log record for that transaction is not needed anymore and its space in the log file can be reused.

To accommodate for reuse, SQL Server organizes the log files as a ring buffer of several containers called virtual log files. There is no direct way to influence their size and or number. SQL Server manages that automatically.

The virtual log files keep track of all their transaction log records and note if they are still needed, for example in an open transaction. Once all log records within a virtual log file are not used anymore, the virtual log file itself is marked as ready for reuse, and SQL Server will overwrite it with new log records once it gets around to that place in the ring buffer. The process of marking one or more virtual log files as reusable is called log truncation.

Log Reuse Wait

The above section described the general behavior of log reuse. There can however be several reasons for a log record to still be required by SQL Server for (potential) future operations. With that its virtual log file cannot be reused. If that happens for an extended period of time, SQL Server might run out of virtual log files and has to add additional ones. For that the physical file has to grow. If autogrowth is enabled for the log file and there is enough room on the drive this will happen automatically. If automatic growth is not possible, the database becomes effectively read-only, causing all write attempts to fail until the situation has been resolved.

The query shown at the beginning of this article allows us to find the most prevalent reason why virtual log files can't currently be reused. As of SQL Server 2012 it can return 10 different values:

• NOTHING
• CHECKPOINT
• LOG_BACKUP
• ACTIVE_BACKUP_OR_RESTORE
• ACTIVE_TRANSACTION
• DATABASE_MIRRORING
• REPLICATION
• DATABASE_SNAPSHOT_CREATION
• LOG_SCAN
• OTHER_TRANSIENT

The first one, NOTHING, means that there are still free virtual log files available. The last one, OTHER_TRANSIENT, is currently not used. That leaves eight real reasons why your log file might be growing.

Summary

SQL Server transaction log files are organized as a ring buffer of log record containers called virtual log files. These virtual log files are reused as the file pointer loops around the ring buffer. However, there are eight reasons that can prevent the reuse of these virtual log files. If reuse is not possible for one of those reasons, the log file has to grow.

Log Reuse Wait Series

Over the next few days I am going to write in more detail about each of the eight log reuse wait reasons.
Below is a list of links to the posts that are already available.