I recently installed Mailman with Exim4, which was a challenge considering all the manual configuration you have to do. I found a few guides on the install, but they didn’t seem to be “complete” enough to get the system working. Turns out, the biggest challenge was Exim4. The configuration files are confusing, especially since there are two sets of configuration files.

One thing I’ve encountered with Awstats is adding in old apache log files. This is useful if you are migrating data from servers. By default, Awstats will ignore ALL past dates in the log that occur before the most recent log entry date. In order to avoid Awstats from finding the “recent log entry date”, you will need move all Awstats cached files into a separate folder.

Here’s a quick tip to make cleaner looking URLs in CodeIgniter PHP framework. By default, if you have an application, the path of the application usually ends after index.php. Using Apache RewriteEngine, you can make cleaner looking URLs. You can have something similar to mysite.com/cigniter/MyApplication instead of mysite.com/cigniter/index.php/MyApplication

I recently implemented a way to limit access by IP range on specific blogs on WordPress MU. As you know, WordPress MU uses Apache Rewrite engine to rewrite URLs. For instance, you have a blog on WordPress MU called “intranet”. Apache Rewrite takes the “intranet” string in the URL and automatically rewrites it as a value in the PHP script. A side effect to Apache Rewrite is that “Directory” .htaccess parameters don’t work. So, if you wanted only your company IPs to access an internal blog, you will need to use Apache Rewrite parameters instead. For the solution, read on…

Sitemaps are useful if you want search engines to look in specific directories of your website. The standard robots.txt notation only has the exclusion list; where not to look and the search frequency.

For instance, a really basic robots.txt file looks like this:

User-agent: *
Crawl-delay: 3
Disallow:/cgi-bin/


For me, I set the Crawl-delay to 3 as a general rule to prevent crawlers from consuming all the web server bandwidth. Generally, Yahoo crawlers are the most aggressive on your site, Google averages about ~13 seconds per request. Anyway, a sitemap gives the crawler a better idea of where to search, rather than trying to discover …

The fastest way to trim down large web log files is through UNIX/Linux shell. Large files exceeding 1 GB (millions of lines of logs) are not easily editable using a GUI interface, so the fastest way is to parse them is via command line. You can trim them down according to a time range, remove internal requests from within the company, and remove bots/crawlers data from the log files.

For instance, I have a 4GB log file with about two years worth of info (2007-2009) in there. What if I just wanted the logs from 2008? First, run the “head -10″ command …

If you want to remove image files from your awstats reports, modify the SkipFiles variable in the /etc/awstats/awstats.conf (or awstats.YOURHOST.conf)

Do a search for “SkipFiles” in the file using nano/vi/emacs and find the section that talks about “Use SkipFiles to ignore access to URLs that match one of the following entries…” The SkipFiles line should look similar to the following:

SkipFiles="REGEX[.jpg$] REGEX[.gif$] REGEX[.png$]"


Installing awstats in Debian with GeoIP caching

First, use apt-get to get the software:

apt-get install awstats

Configure awstats:
Instead of using the awstats configure tool (written in Perl), manually add in the awstats Apache configuration. Using the tutorial from debuntu, make a file inside your apache config folder called awstats.conf with the following lines.

Alias /awstatsclasses "/usr/share/awstats/lib/"
Alias /awstats-icon/ "/usr/share/awstats/icon/"
Alias /awstatscss "/usr/share/doc/awstats/examples/css"
Options ExecCGI -MultiViews +SymLinksIfOwnerMatch

Inside the Apache2.conf file (or virtual host conf file inside the sites-available folder) add this to the end of the file:

Include /etc/apache2/awstats.conf

Next, copy the /usr/lib/cgi-bin/awstats.pl file to the apache cgi-bin folder. If you have virtual hosts enabled, copy it to …