AWStats: Squid Log Analyzer
Configuring Squid proxy cache with AWStats on Windows Contents
1 Overview
2 To configure your machine to use the proxy
3 To use the reporting tool (after its all set up)
4 Security
5 The setup: Server installation
5.1 Download and install ActivePerl
5.2 Notepad++
5.3 Download and install Squid
5.4 Configuration changes to squid
5.4.1 Log files
5.4.2 Logging format
5.5 Get perl working in IIS
5.6 Download and set up an AWSTATs server
5.7 Configure AWStats
5.8 Give AWStats something to chomp on
5.9 Maintenance
5.9.1 Log file size
5.9.2 Update history size
5.9.3 Setting the schedule
6 Appendix A : All urls used in research
7 Appendix B : Commands from article
1 Overview
This article serves as a step by step overview and reminder for how we established a squid proxy cache on our Windows network, along with AWStats for the security department to be able to audit and review our Internet traffic.
2 To configure your machine to use the proxy
On a Windows 10 machine.
Windows in the bottom left of the screen > choose “Settings” > “Network & Internet” > “Proxy” Use a proxy server : On Address : 192.168.0.254 Port : 3128 Don’t use the proxy server for local (intranet) addresses : Tick the box
In the future if you don’t use the proxy your machine won’t get out to the internet.
3 To use the reporting tool (after its all set up)
You must be inside the network!
http://192.168.0.254/awstats.pl?config=allnone.ie
At the top is an “update now” button. Click that to update the logs.
4 Security
There is a very basic authentication service possible on the proxy
C:\Squid\etc\passwd
That is a file that can be opened in notepad.
You add one line per user
You can use the following tool to generate the line that goes into the file
http://www.htaccesstools.com/htpasswd-generator-windows/
In the future we’ll want to link it to AD, but this involves samba / ldap / and other stuff a bridge too far for right now.
Add users with test passwords all you want.
Once everyone is going through the proxy, you then put rules in place on the firewall to block access externally, which means you have to go through the proxy to get out.
5 The setup: Server installation
5.1 Download and install ActivePerl
Usually goes into C:\Perl64\
Make sure perl works from the command prompt (i.e. it got added to the environment path)
5.2 Notepad++
If notepad++ isn’t on the machine install that as well.
Change the defaults for .conf, .txt, .log to go to. Saves loads of hassle.
5.3 Download and install Squid
Just follow the default procedures available and take note of
Install directory
Basic config changes
http_port
dns_nameservers
Allow ports and ensure DNS servers are accessible.
5.4 Configuration changes to squid
5.4.1 Log files
Add to the squid.conf file the following lines
access_log C:\Squid\var\log\squid\access.log combined cache_log d:\Squid\log\cache.log
That means squid logs in the directories specified. Great for saving your poor space starved C: drive.
5.4.2 Logging format
Add the line
logformat combined %>a %ui %un [%{%d/%b/%Y:%H:%M:%S +0000}tl] "%rm %ru HTTP/%rv" %Hs %<st "%{Referer}>h" "%{User-Agent}>h" %Ss:%Sh
That gives a format that AWStats can read
5.5 Get perl working in IIS
This allows perl scripts to be run through IIS
http://www.howtogeek.com/50479/how-to-install-perl-on-iis-7-for-windows-server-2008/
You end up with the commands being slightly off, so this can help
https://forums.iis.net/t/1171475.aspx
This got it sorted for me
https://community.activestate.com/forum/activeperl-iis-75
5.6 Download and set up an AWSTATs server
The primary config approach. Be very careful… all domains are case sensitive though not observed in the supplied commands. Stick everything in lower case always
http://blogs.iis.net/steveschofield/getting-awstats-installed-and-configured-on-iis-7-5
After this you need to also copy in the cgi-bin folders (lang lib and plugin really need to not be in that sub dir)
Note test.pl is from the testing Perl In IIS from the previous setup article and could be removed.
http://www.howtogeek.com/50526/setting-up-awstats-on-windows-server-and-iis/
Doesn’t give much bar the really useful line of how to start the thing
http://localhost/awstats/awstats.pl?config=domain.com
5.7 Configure AWStats
Location C:\inetpub\stats.allnone.ie and the file awstats.allnone.ie.conf which is created from the model example one provided.
LogFile="D:\Squid\log\access.log" LogType=W LogFormat=1 SiteDomain="stats.allnone.ie" HostAliases="localhost 127.0.0.1 REGEX[stats.allnone\.ie$]" DNSLookup=2 DirData="C:\inetpub\stats.allnone.ie\stats\Domain1" DirCgi="C:\inetpub\stats.allnone.ie\" AllowToUpdateStatsFromBrowser=1
There is security to limit access to the report, but we’re not to that point yet.
5.8 Give AWStats something to chomp on
So to generate the file…. Create a batch file
C:\Perl64\bin\perl.exe C:\inetpub\stats.allnone.ie\awstats.pl -config=allnone.ie
That should create a awstats042016.allnone.ie.txt file in the C:\inetpub\stats.allnone.ie\stats\Domain1
That process creates a raw initial comprehensive single file.
Update from within the browser will “add to the history”. So to keep the report file smaller… we use this batch file.
5.9 Maintenance
5.9.1 Log file size
The log files are going to get rather big, so we need to rotate them with a simple dos command in a batch file nightly.
squid -k rotate
5.9.2 Update history size
As mentioned in the previous step. A simple recreate of the file nightly.
5.9.3 Setting the schedule
The batch files are in D:\Maintenance. Both to run nightly just before midnight.