My Techie-Thoughts

Most every program you install and run, especially services, generate some form of log file and nearly everyone only checks those logs when something bad happens.  Why? Because there are so many logs to check!  Well, that’s where a log parsing program can be a lifesaver.  I like using Logwatch on my Debian/Ubuntu systems. Logwatch is a nice, lightweight, easy-to-use program that generates a summary report that can be emailed nightly or on whatever schedule you choose.

Notice that I said summary?  Yes, that’s important.  It means that you can configure it to just give you a quick overview of essential processes such as automatic updates, cronjobs, backups, webserver errors, attempted hacking attempts, etc. instead of having to manually review all those logs separately and in excessive detail – which is the reason you avoid this very important task in the first place.

How it works…

Logwatch comes preconfigured to monitor a very wide variety of logs, so chances are you can use it straight out of the box with minimal configuration.  You need to let it know where your logs are stored (default: /var/log/ on Debian/Ubuntu systems).  Then it runs those logs through a series of simple and customizable PERL filters to create a summary report that is separated by headers for each monitored service and saved or emailed as a text or HTML file.  Pretty simple, right?  Let’s get it set up.

Installing Logwatch

For most people, you will want your reports emailed to you.  Logwatch will automatically install EXIM4/Sendmail to use as an MTA if necessary, but that is wayyyy too much work to get configured and working properly.  Check out my previous article about a simple setup for email notifications on a linux system using msmtp and get your MTA sorted out before continuing. Installation is very simple:

apt install logwatch

Here’s the part that a curious amount of other blog entries about installing Logwatch forget to include: You must copy the configuration file to /etc/logwatch/conf/ and then edit it, otherwise your settings will not be applied!

cp /usr/share/logwatch/default.conf/logwatch.conf /etc/logwatch/conf/

Basic configuration

You’ll have to open up that /etc/logwatch/conf/logwatch.conf file in your favourite text editor, I like nano but you can use whatever.  Let’s go through the basic changes that you should make to get started:

• LogDir: The default, /var/log, is likely correct for most systems, but double-check it to be sure.  This is where Logwatch will search for application log files by default unless told otherwise by a specific service file.
• TmpDir: This can be any directory you want, but /var/cache/logwatch is a pretty good fit for most installs.  You’ll have to manually create this directory when we’re done editing this file, so make a note of this location.

• Output: By default, this is set to stdout.  That’s not really helpful unless you plan on watching your screen every time a report is issued.  You can choose file or mail:

  • If you choose file, you’ll want to set the following parameters:
    

  Output = file
  Filename = /path/to/file

  • If you choose mail, set your parameters as follows:
    

  Output = mail
  MailTo = "Firstname Lastname" <[email protected]>
  MailFrom = "Optional Name" <[email protected]>

• Format: By default, Logwatch will generate plain-text output.  If you’d prefer HTML, set it that way:

Format = html

• Range: This is the default reporting period.  Usually, you’d want to see yesterday’s activities.  We’ll cover a weekly range later if you’d prefer that.

Range = yesterday

• Detail: This is dependant on the filters being used, but the default setting of Low (or 0) is generally sufficient, so we’ll leave this alone.
• mailer: This is very very likely ok to leave at the default, sendmail.  If you used my recommended msmtp-mta setup, then the default is fine.  You only have to change this if you know you are using an MTA that is not sendmail-compatible.

One more step and we’re done.  Remember I said something about that ‘TmpDir’ setting?  You need to make that directory for Logwatch since it won’t do that itself for some reason.  Assuming you accepted the default:

mkdir /var/cache/logwatch

Testing

Let’s take Logwatch for a quick test-run to make sure we didn’t do anything weird in our setup.  Since we already set our defaults in the previous section, testing becomes really easy:

sudo /usr/sbin/logwatch

If you wanted to test different parameters, you can specify those on the command line to override your default settings.  For example, if I wanted the output to be on the screen instead of via email but otherwise keep everything else at my defaults, then I’d run:

sudo /usr/sbin/logwatch ––output stdout

You can get a full list of options by passing the ‘–help’ parameter to Logwatch.

Scheduling

Logwatch creates a default cronjob at /etc/cron.daily/00logwatch which is set to run at the default cron.daily time for your system (06:25 on most Debian systems) and send an email as configured in your logwatch.conf.  If that works for you, then you can leave this as-is.  Otherwise, follow me :-)

First, let’s disable the default cronjob by removing the execute permissions:

sudo chmod a-x /etc/cron.daily/00logwatch

Now, let’s set up our root user to run logwatch as we have configured it in our logwatch.conf.  Let’s start by accessing our root user’s crontab:

sudo crontab -e

Now, let’s set up a cronjob.  In this example, I’ll add an entry to run Logwatch at 3:17am everyday:

# run logwatch at 03:17 daily
17 3 * * * /usr/sbin/logwatch > /dev/null 2>&1

Just like when we were testing, we don’t have to pass any command-line parameters because we changed the default configuration file.  If you want to use different parameters, just add them as needed.  The ‘> dev/null 2>&1’ at the end means ‘redirect any output and errors to null’ since there is no one around to see that output and we don’t want to junk up our console logs.

Weekly output

I know a lot of sysadmins and most regular people find daily log reports a little much.  Even the nice summaries provided by Logwatch still add up if you have several servers you have to keep an eye on or if you have many services generating log files on just a single server.  Logwatch only understands ‘range’ parameters of ‘yesterday, today, all’ by default, so we have to get a little creative.  First, we have to understand that basically everything about Logwatch is PERL based, so let’s start there. Grab the Date::Manip date manipulation module so PERL can understand more complex date/time stuff (it’s easier to run all these commands as your root user so you don’t have to ‘sudo’):

apt install libdate-manip-perl

Perfect. Now let’s adjust our Logwatch defaults. Open your /etc/logwatch/conf/logwatch.conf and change the Range parameter to the following:

Range = between -7 days and -1 days

Finally, change your cronjob to make sense with your newly established date range.  If you’re using the default cronjob that was installed with Logwatch, then just move it from /etc/cron.daily/ to /etc/cron.weekly/:

mv /etc/cron.daily/00logwatch /etc/cron.weekly

If you’re using a custom cronjob run by your root user, then open up root’s crontab as explained earlier and update the crontab entry to something like (notice the * * 0):

# run logwatch at 03:17 every Sunday
17 3 * * 0 /usr/sbin/logwatch > /dev/null 2>&1

That ‘0’ at the end means ‘Sunday’ (days are represented by numbers 0-6, with Sunday=0, Saturday=6).  So Logwatch will run on Sunday at 3:17am and will include a range of everything between 7 days ago (-7 days) and yesterday (-1 days).  You can play around with these settings to suit your preferred schedule.

Final thoughts…

Well, that’s Logwatch in a nutshell.  It’s pretty easy to set up, very reliable and allows you to keep an eye on your logs without having to wade through numerous files.  I’ll do a series on customizing Logwatch if you want since it’s a pretty powerful tool that I use to keep an eye on logs generated by my custom scripts too.  Let me know if that’s of interest in the comments.