Log Rotation in Linux

A normal linux system generates several logs in its day-to-day operations.Many of these logs may be generated by applications/packages installed on the system.It is, therefore, essential to manage these logs.Linux has an inbuilt logrotate utility to simplify the administration of log files on a system which generates a lot of log files.Logrotate allows for the automatic rotation compression, removal and mailing of log files.Logrotate can be set to handle a log file daily, weekly, monthly or when the log file gets to a certain size.

Before we understand how logrotate command works,here is a list of some important files 

/usr/sbin/logrotate - The logrotate command itself

/etc/cron.daily/logrotate - The shell script which executes the logrotate command daily.

/etc/logrotate.conf - Log rotation configuration for all the log files are specified in this file.

/etc/logrotate.d - When individual packages are installed on the system,they drop the log rotation configuration into this directory.

Log rotation for a file

If you want to rotate a log file (example- /tmp/example.log) for every 10 kb, include the following in the /etc/logrotate.conf file

$ cat /etc/logrotate.conf

# Now include this snippet in the file

/tmp/example.log {
size 10k
create 0644 root root 
rotate 4
}

The above configuration has the following three options

size 10k - Logrotate runs only if the filesize is equal to (or greater than) 10 kilobytes. You can change or set this filesize limit to whatever you want.

create - Rotate the original file and create the new file with specified permission,user and group.(Here 0644 indicates -rw-r--r-- file permission,user is root and user group is root.Do ls -l filename to find out this info about any file on your system.)

rotate 4 - Limits the number of log file rotation.So,this would keep only the recent 4 rotated log files.If it is set to 0, old versions(of files) are removed rather than rotated.

Before the logrotation,the size of the output.log file is shown below(just an example)

$ ls -l /tmp/example.log

-rw-r--r-- 1 root root 25868 2012-09-23 16:54 /tmp/output.log

Now execute the logrotate command(yes, we run it manually this time around although the logrotation is generally an automated task.See /etc/cron.daily/logrotate)

$ logrotate -s /var/log/logstat logrotate.conf 

Option -s specifies the filename to write the logrotate status.Here we write it into a file named logstat.This file if present,the logrotate writes the status information into it, otherwise,this file is automatically created on execution of the above command.Once you execute the above command,the logrotation status is written into /var/log/logstat. You can cat this file to view the information.

This was an example to demonstrate how logrotation works.There are several other options for logrotaion.Some of them are explained below

• Use the compress option in the logrotate configuration file to compress the rotated files.The files will be compressed with gzip utility. 

• Use the dateext option in the logrotate.conf to rotate the old log file with date in the log filename.However,this option should be used only if logrotation is to be done only once a day.This is because if you try to rotate logs more than once on the same day,the earlier rotated file will be having the same filename.So logrotate won't be successful after the first run on the same day.     

When using the above two options in the logrotate configuration,the rotated file name will be something like this
example.log-20120923.gz

Rotate logs monthly,daily or weekly

For doing the log rotation monthly once use the monthly option(keyword) in the logrotate.conf file along with the snippet shown above.Similarly, add the weekly/daily keyword for weekly/daily log rotation.

Logrotate postrotate endscript option:Run custom shell scripts immediately after log rotation

Logrotate allows you to run your own custom shell scripts after it completes the log file rotation.The following configuration indicates that it will execute exscript.sh after the logrotation

$ cat /etc/logrotate.conf

# Now include this snippet in the file

/tmp/example.log {
size 10k
create 0644 root root 
rotate 4
compress
postrotate
    /home/siddharth/exscript.sh
endscript
}

Some other useful options

Use maxage option to remove older rotated log files after a specific number of days.For example including maxage 100 in the logrotate.conf file indicates that the rotated log files would be removed after 100 days.

missingok option - You can ignore the error message when the actual file (to be rotated) is not available by using this option.

mail address - When a log is rotated out-of-existence,it is mailed to the 'address' specified with this option.

For additional help i am including a sample of logrotate.conf file for the above example and option(s) described.