Shane Rainville
I.T. professional with over a decade of experience, ranging from application development to system & infrastructure administration. He's worked with small startups to large corporate companies, using unique and creative solutions to solve problems.

Overview

It’s always important to keep your server logs around for as long as it makes business sense. You’ll need them for auditing system access, discovering abuses, or to identify root causes to problems, among other reasons. The challenge, though, is that depending on the service being provided and the amount of traffic received, your logs are capable of growing to gargantuan sizes, consuming every last bit of disk space available.

Logrotate allows us to better manage our logs to prevent from consuming too much disk space. Depending on the schedule you decide on, your logs can be rotated every day, week, or month. Each rotation renames your existing log file, usually by appending a ‘.’ and number to the end, and then creates a new file. To preserve storage the logs that have been rotated can be compressed using Gzip.

Thankfully, most major Linux distributions, like Ubuntu, automatically rotates most logs found under /var/log.

Installation

Logrotate is installed by default on most recent distributions.  As of this writing that means Ubuntu 15.10 and CentOS 7 – 1511. However, if you find that your installation does not include it, it can easily be installed through Yum or Apt.

Create a Log Rotate Configuration

Logrotate configuration files are stored under /etc/logrotate.d. When the logrotate cron job runs, it will look execute any configuration found in that directory.

The following example is that of a default Apache2 log from a CentOS 7 server. We can use it as a template for our own application logs, separate from the default ones.

/var/log/httpd/*log {
    missingok
    notifempty
    sharedscripts
    delaycompress
    postrotate
        /bin/systemctl reload httpd.service > /dev/null 2>/dev/null || true
    endscript
}

Here’s an explanation of the settings used above.

/var/log/httpd/*log The location of the log file(s). In this example, find any file that ends with ‘log’ in the /var/log/httpd directory.
missingok Tells logrotate to ignore any errors if the log file(s) does not exist.
notifempty Do not rotate log file if it is empty.
sharedscripts The posrotate script will only run once, after all matching log files are found. If one log file doesn’t need rotating, the script will not run.
delaycompress Delay compression of a rotated log file until the next rotation. This allows us to keep the last rotation uncompressed and easily searchable, while ensuring older rotations are compressed.
postrotate The command executed after rotation is complete. In our example above, Apache (httpd) is restarted, which causes it to create new log files.

Logrotate supports many more options than what’s listed above. To see a complete list, you can man logrotate or visit linuxcommand.org logrotate page. Here are a few options you may find useful.

daily Rotate the logs daily.
weekly Rotate the logs weekly.
monthly Rotate the logs monthly.
rotate x Rotate the log x amount of times before purging older logs. Using ‘rotate 5’ means you will only keep 5 rotations of a log.
copy This option copies the log file, essential creating a snapshot of it at a particular point in time.
prerotate Like postrotate, but runs a script before log rotation is completed.

 

Test Your Configuration

As with anything, before you roll your configuration into production you will want to test it to ensure everything works. Logrotate includes a feature that allows us to run a configuration, in debug mode, without it doing any work. If there are errors discovered in the syntax or some other issues, you will be notified.

sudo logrotate -d /etc/logrotate.d/myapp.conf

The output will look similar to the example below. The output displays the discovered logs based on the location /var/httpd/*log, and what was done for each log. None of the logs were rotated since I ran the test on a server that was newly built. However, if the logs had more content and were aged a little more, we would see a message indicating our logs were rotated. For now, we can take this as our configuration file syntax is correct.

empty log files are not rotated, old logs are removed
considering log /var/log/httpd/access_log
  log does not need rotating
considering log /var/log/httpd/error_log
  log does not need rotating
not running postrotate script, since no logs were rotated

© 2014 Shane Rainville