Monday 21 July 2014

Managing Logs in Apache || Setting Log Rotation Policies for Apache Web Server

Hi everyone,

I just started to work for jboss application server. In my environment I have jboss application server configured with Apache web server. When first time I checked my apache server log I found that size of my access.log file is more than 1GB and I was unable to open it. When I was looking for the solution I came across the term log rotation.

For managing logs in apache you need to rotate the logs either on time or size basis. In my case I decide to rotate my error and access logs after every 20MB so I can easily read and analyze it. Let's look at how to make logging more easily manageable in Apache.

Before we start let us take a look at some of the Apache logging directives that are commonly used. These can be used to change the way logging works in Apache.

Logging Directives

  1. CustomLog: The CustomLog directive is used to set the log file name and its format. This directive is used to configure the access_log logs.
  2. LogFormat: This directive is used when you want to set the format for the access logs.
  3. LogLevel: This directive used to set the level of verbosity in the error logs.

Configure Logging

Configuring Apache logging to make it easier to manage can be done in several ways. The Apache configuration file, usually located at /etc/httpd/conf/httpd.conf, has a section that defines some basic logging in the default configuration file that ships with an Apache installation.

Set the LogLevel:

This should generally be set to warn. You can set this to a higher level of verbosity on a development machine as you might want more information from the server. However, if you are running a production server you want to avoid creating too many logs as that will slow down the server.

LogLevel warn

Some of the other useful options for setting the level of logging are Emerg for emergencies, Crit for critical issues, Error to set it to error level, which is a little less verbose than warning level. The two most verbose options are Info and Debug. These write a lot of data, and can eat up space very quickly.

Set the LogFormat parameter:

This is the directive used to set the format of the logs that are generated. You can set the user agent and other details of the log files. Note that you should avoid tweaking this unless you know what you are doing. This is especially the case if you plan to use a standard log file parsing tool to analyze your web server logs.

LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent 

Configure Error Logs:

As discussed earlier, there are two important logs created by default. The error log is one of them. This is the file where your web server writes all of the issues it encounters. This log can be a lifesaver when troubleshooting an issue with the server. The simplest way to configure the error log's parameter is to set it to something like this:

ErrorLog /etc/httpd/logs/error_log

However, if you want to manage your logs in a smarter way, there are two things that you might want to use. The first is log rotation, and the other is setting up log file names to change on an hourly basis. The log rotation can be handled by a tool that ships with most Apache installations, called rotatelogs.

Setting up the logs to create 20MB size file is a simple hack. For those familiar with the date command in Linux, the Apache configuration file understand and translate some of the date commands. It interprets %Y as year, %m as month, %d as date, and %H as the hour of the day, %M as minutes and %S as seconds. We use this parameter together like %Y-%m-%d-%H_%M_%S to set naming convention and track the exact timing. And for the file rotate after every 20M simply write it after the naming conventions. The final error log's configuration would look something like this.

ErrorLog "|bin/rotatelogs logs/error.log.%Y-%m-%d-%H_%M_%S 20M"

This configuration will rotate the error logfile whenever it reaches a size of 20 megabytes, and the suffix to the logfile name will be created of the form errorlog.YYYY-mm-dd-HH_MM_SS.

Access Logs:

The other important log generated by Apache is the access log. This is the file where Apache logs information on every visitor and visit to the web server. This is extremely useful to track and monitor usage patterns, track malicious users, and basically keep an eye on proceedings. As your usage increases, so will the size of this file. So again, like the error file, it is a good idea to set the rotation policy we used for the error logs for the access logs as well. We use the same rotation policy and setup the access logs to look something like this.

CustomLog "|bin/rotatelogs logs/access.log.%Y-%m-%d-%H_%M_%S 20M" combined

This configuration will rotate the access logfile whenever it reaches a size of 20 megabytes, and the suffix to the logfile name will be created of the form errorlog.YYYY-mm-dd-HH_MM_SS.
 
After making changes in httpd.conf file it is good to take a restart of apache server to activate those changes. And now your access and error logs are rotating after every 20MB and will create a new file.

You can find more information on its usage of rotatelogs at link http://httpd.apache.org/docs/2.0/programs/rotatelogs.html 

It works in my case hope this will help you.

Effort only fully releases its reward after a person refuses to quit.”

Regards,
Akhilesh B. Humbe

No comments:

Post a Comment

Popular Posts