Using Logrotate To Manage Log Files

Introduction

Logrotate is a Linux utility that simplifies the administration of log files. It typically runs once a day via a cron job, and manages logs based on customized rules/settings from its config file.

Some of its useful settings include automatic rotation, compression, removal, and mailing of log files.

Installation

Most Linux systems come with Logrotate installed by default. Check if you have it installed on your Vultr instance by issuing the logrotate command. You will see an output with the version of Logrotate that your server is running.

If you don’t have it installed, perform the steps below to proceed with the installation.

On Debian/Ubuntu systems:

sudo apt-get update
sudo apt-get install logrotate

On Redhat/CentOS systems:

sudo yum update
sudo yum install logrotate

Understanding Logrotate

The moving parts that make logrotate run are:

• The actual tool logrotate.

• Logrotate’s configuration file located at /etc/logrotate.conf. This file holds the configuration for all log files that Logrotate manages.

• A daily cron job /etc/cron.daily/logrotate that issues the logrotate command to run based on settings in its configuration file. If this cron job does not exist on your system, create it and add code snippet below to it.

  #!/bin/sh
  /usr/sbin/logrotate /etc/logrotate.conf
  

More on Configuration

If you take a peek inside /etc/logrotate.conf, you will see that it has the line include /etc/logrotate.d in it. What this line does is tell Logrotate to look inside the /etc/logrotate.d directory and run every configuration file in it. This directory is typically where applications installed on your linux system will add their logrotate configurations. For example, Apache2 will typically create a /etc/logrotate.d/apache configuration file upon installation.

Managing Logs

To test Logrotate, we will:

• Create a new test log file with 1MB or random data:

  sudo base64 /dev/urandom | head -c 1000000 > /tmp/testfile.log
  

• Create a new Logrotate configuration by running sudo nano /etc/logrotate.d/testlog. Copy the following snippet into empty file and hit Ctrl + X to save and exit.

  /tmp/testfile.log {
      size 1k
      copytruncate
      missingok
      rotate 5
  }
  

The configuration options in the snippet above instruct Logrotate to:

• size 1k: Rotate log file if size is greater than or equal to 1k.
• missingok: Ignore error messages if testfile.log does not exist.
• copytruncate: Create a copy of current log file and then truncate it. This comes in handy when an application cannot close its log file because it continuously appends to it.
• rotate 5: limit the number of log file rotations to 5. This will delete old versions of log files greater than 5 days.

NOTE: You can see all configuration options for logrotate here.

• Run the Logrotate command manually:

  sudo logrotate /etc/logrotate.conf
  

  After running above command, list all files in tmp directory ls -l /tmp to confirm testfile.log was indeed rotated. You should see a listing similar to the following that shows testfile.log has indeed been rotated. This will continue every day and keep the latest 5 copies.

  [root@vultr ~]# ls -l /tmp
  -rw-r--r--  1 root root       0 Nov 14 23:31 testfile.log
  -rw-r--r--  1 root root 1000000 Nov 14 23:30 testfile.log-20141114
  -rw-r--r--  1 root root     634 Nov 10 00:23 vultr_ipv6
  -rw-------. 1 root root       0 Oct 15 20:44 yum.log
  

Conclusion

Logs can quickly create problems on a server by becoming too large and causing disk space issues. Managing logs is crucial for any system, but even better is automating this management. Logrotate makes rotating, archiving, and deleting logs easy.

Written by Lami Adabonyan