HOWTO: Install awstats on Textdrive

While we are all waiting for Anemone/Textstat/whatever to emerge, the need for a log based stats tool persists. Slimstat/Mint/Urchin are fine, but they all have similar limitations: since they require inserting some code (PHP/Javascript) in the source to be monitored, they can only keep track of limited types of content. In particular, they can’t keep track of non-HTML content (well, there are ways to hack around this, but I’m talking out of the box) such as images, downloadables and so on. Even more importantly, they have no way of reporting figures like total bandwidth consumed, which is critical in a shared hosting environment such as Textdrive.

There’s already one howto for installing Awstats on Textdrive over at the Textdrive forums. But I found the howto needlessly long and complicated. I managed to install Awstats in a (hopefully) much simpler way, and this howto is to share that information.

Step 1: Download Awstats

Download the latest version (currently 6.5) from the Awstats website or head to the project page on Sourceforge. I’ll assume that you have downloaded the tar.gz in ”~/tmp” on your Textdrive account. Substitute ”username” with your own username where applicable.

Extract the files:

$ pwd
/users/home/username/tmp/awstats-6.5
$ tar -zxf awstats-6.5.tar.gz

Step 2: Setup files

Move the executable scripts to ”cgi-bin”:

$ cd awstats-6.5
$ mv wwwroot/cgi-bin ~/cgi-bin/awstats

Move the rest of the files to your web root:

$ mv wwwroot ~/web/public/awstats

Create directory to store awstats data:

$ mkdir -p ~/homes/awstats-data

Step 3: Configuring Awstats

Copy over the model configuration file:


$ cd ~/cgi-bin/awstats
$ cp awstats.model.conf awstats.yourdomain.com.conf

Modify the newly copied configuration file using your favorite text editor:

  • Set ”LogFile = “/users/home/username/logs/access_log””
  • Set ”LogFormat = 1”
  • Set ”SiteDomain = yourdomain.com”
  • Set ”DirData = “/home/username/homes/awstats-data””
  • Set ”DirCgi = “/cgi-bin/awstats””
  • Set ”DirIcons = “/awstats/icon””
  • Set ”StyleSheet = “/awstats/css/awstats_default.css””

The above LogFormat setting assumes that you’re access log files are set to use the extended combined log format. This should typically be the deafult. If it isn’t and you are unsure how to do this, take a look at this forum post.

Test that it works:

$ cd ~/cgi-bin/awstats
$ ./awstats.pl -update -config=yourdomain.com

You should see awstats spew some output saying how many records were processed etc. Unless you see some error, you’re good to go.

Step 4: Setup cron job

The main rules here are:

  • Don’t run your cron job more than once a day
  • Don’t run the awstats update script on more than one log file at a time

Your access logs are rotated on a daily basis at around midnight. So I have my cron job to simply scan the log file 5-10 min before midnight. That way, I process only one logfile at a time, and I (hopefully) don’t miss too much data (with this setup, awstats won’t log hits that you get during that 5-10 min window).

Here’s how to do it:

  • Log into webmin
  • Goto “Scheduled Cron Jobs”
  • Click on “Create a new scheduled cron job”
  • For “Command”, enter ”/users/home/username/cgi-bin/awstats/awstats.pl -update -config=yourdomain.com”
  • Leave “Input to command” empty
  • For description, enter “awstats” or some such
  • Select “Times and dates below” for “when to execute”
  • leave “Days”, “Months” and “weekdays” at “All”
  • For “Minutes”, choose “selected” and click on **one of 45/50/55**
  • For “Hours”, choose “selected” and click on 23
  • Click on create

Thats it! You should be all set now. Try to check out your stats at ”http://yourdomain.com/cgi-bin/awstats/awstats.pl?config=yourdomain.com”

Beyond the basic configuration

You might want to secure your setup so that only you can view your stats. I just followed the instructions in [[http://kb.textdrive.com/article/password-protecting-a-directory-with-digest-authentication-doesnt-work-it-keeps-kicking-me-out|this Textdrive KnowledgeBase article]] and it worked perfectly fine for me. Just create the ”.htaccess” file in ”~/cgi-bin/awstats”

If you are setting up Awstats for the first time, you might be having a lot of old log file lying around in ”~/logs” that Awstats hasn’t looked at yet. Ideally what you would like is to feed Awstats all the old log files so that you can look at your stats since back then. Here’s a simple shell script to do this — basically it feeds the log files one at a time to Awstats, in chronological order. Be sure to use liberal sleep values so that the process limiting mechanisms in place don’t kill it:


for log in $(ls -rt ~/logs/access*.gz)
do
echo $log
echo "zcat"
zcat $log > ~/tmp/access_log
sleep 30
echo "awstats.pl"
./awstats.pl -update -config=yourdomain.com -LogFile=/users/home/username/tmp/access_log
sleep 30
done

Before you run the above script, make sure that you delete all files under ~/homes/awstats-data. It seems that Awstats needs to be fed data in chronological order starting from a clean state, otherwise it gets confused.

Posted in

5 comments

  1. Pingback: Floating Sun » HOWTO: Install awstats on Textdrive
  2. Diwaker Gupta

    @Hypnotherapy Australia: Thanks, glad you found it useful :) Let me know if you have any comments or suggestions to improve it. I haven’t revisited it in a while, so if there’s something that doesn’t work anymore or needs updating, drop me a note!

Leave a Reply