This section will teach you how to use W3Perl. First, check that your system has
everything to run this package. An installation manual is available
for each platform.
You can install the package without telnet access (FTP) nor CGI
available. A Web adminstration interface allows you to manage your
stats but all scripts can be run using command lines. For easy
installation, RPM is available for Linux and there is a .exe file for Windows.
Informations are available on the configuration files to
customize your output, choose the right logfile format.
You can run the package from the web administration interface or from
a crontab.
Don't mix both
as web user has very limited privileges and
won't be able to read/write file owned by a user.
W3Perl provides the usual stats as
Pages,
Hosts,
Countries,
Directories,
Download,
Hourly,
Daily,
Weekly,
Monthly,
Referer,
Browsers,
Traffic,
Search engine,
Error,
Scripts
... but many more as
Real-time,
Session,
Virus,
URL mapping ...
If you want to upgrade, see first the latest news about the
package and read how to make the upgrade.
At last, some informations about the next release and the
package limitations.
To use W3Perl , you don't need to install a lots of package or
external Perl modules. It can be run as a standalone package.
As W3Perl is written in Perl, you should have a copy of Perl on your
system. This is not a problem for unix user as Perl is always
installed. Windows user, you'll have to install StrawberryPerl or ActivePerl..
W3Perl use the fly package to build graphics. This is a small tool
which allow the use of the gd library in Perl. W3Perl is based on
the 1.6.5 Fly version as Gif support have been removed from the latest
version.
You can install optional Perl modules which allow you to connect to a
database (DBI) if you're using a CMS as Spip or a module which allow
you to locate IP when the reverse DNS fail.
If your logfiles are compressed, W3Perl need to uncompress them on the
fly so you need at least to have the tool to do the job.
Installing W3Perl is quite easy. You can choose between the RPM
package or the tarball.
RPM is available for Mandrake users. If you want to build your own
RPM, the SRPM is there. If you want to share, send me your RPM so
it will be available to others.
To install W3Perl from the tarball, extract the package inside your
server tree (could be a user alias). Edit the install.pl script to
alter the perl path, your cgi path (if available) and the w3perl path.
Run this script. You can then use the web interface to manage
your stats output or build your configuration files. Some predefined
configuration are available, you can use them as a template.
Once everything is fine, you could either run the stats from the
web interface or from the command lines with cron-w3perl.pl script.
To automatically update your stats, use a crontab to launch this
master script with the incremental mode.
More details are provided in the documentation.
The windows installer have been greatly improved recently. Thanks to
the NSIS tool, installing W3Perl is just a matter of clicking on the
.exe file.
First, the installer will check Perl is available and then
will look for the IIS server. If found, W3Perl install its scripts,
create a virtual directory inside your IIS server and allow this
directory to have executable scripts and finally add the perl mapping
extension. Then the install script is running.
When everything is fine, your favourite browser is call with the
W3Perl web administration page. A default configuration file is
provided for IIS, so you can straight away to launch the stats.
Many things need to be improved as the ability to detect several IIS
Webserver or the Apache server, and then the ability to select on
which servers you want to install W3Perl. Also building configuration
files according to your server by reading the IIS Metadata or Apache
configuration files.
I haven't install W3Perl on a Mac for a while. Some people have done
the job so a little help is available. If you want to add some
comments or update this documentation, you are welcome.
W3Perl can use different plug-in to build extra stats or add new
features.
The first one is the Geo::IP perl module which allow to map IP
to country code, allowing to get country stats with IP logfiles.
It can be used as an alternative to reverse dns as the perl module
use a local database to map IP to country code.
(update are freely available on the maxmind website).
Making DNS queries can be very long, but this is the only way to
resolve IP to hostname, GeoIP doesn't do the same job, IP are not
translated to hostname.
To get cities stats, you can use the GeoLiteCity module from
maxmind which require the GeoIP library.
Stats reports can be exported in PDF thanks to the htmldoc program.
In order to send stats report via email, you'll need to install
the perl MIME::Lite module.
At last, if you want W3Perl to extract data from your SPIP database,
the perl DBI module should be installed.
If you are running SELinux (Security-Enhanced Linux), additionnal customisations
are required in order to run W3Perl. Basically, few chcon commands and some others
security changes.
If you cannot read your server logfile, you can create your own using a javascript code
to be inserted in your pages. It will create a logfile which could be used as an input for W3Perl.
Want to get some stats about your external links ? Install the counter php script
and it will create a logfile for you. Use next W3Perl to analyse this logfile.
The script will redirect your visitor to the target website. It's a good way to
know how many people are leaving your website or if your sponsor links are used.
Configuration files are the way to customize your stats. It includes
how the stats will be build, which stats you want to produce, the
display and many filtering options. A Web Administration interface
is available to help you managing those files. You can create one from
scratch, from a previous template or modify an existing one. Sadly,
people without cgi access won't be able to get this facility, then use
your favourite editor and select an existing one to build your own.
Some predefined configuration are availble for newbies. There are
ready to use with default values. Feel free to change them with the
administration interface.
To create a configuration file, you should first define some basic setup in order to W3Perl to
run : where your logfiles are, which kind of logfile you are using,
if your logfiles are split/compressed, your logfile filename ...
Then the filtering option to select on which directories you want to
get stats, which hosts you want to exclude, the threshold you want
on display ...
And finally, how the stats will be displayed using a background, in
which language ...
You can select how much stats will be output.
If you just need to have
a glimpse, use the lower level and only one page figuring the main
basic stats will be produced. This is the quick way to know the number
of access, hits, hosts...
If you want to see stats versus time, level 2 is the minimum. More
informations will be available but of course it will take longer to
process.
Default configuration is level 3 which give a very good amount of
stats.
At last, if you are not in a hurry and want to have the maximum stats,
select level 4 but you'll need some hard disk space left as the stats
become very precise.
As the logfiles are the input files
for W3Perl, you should carefully select the ones you are using or the
scripts will fail.
Sadly, there are a lot of logfile format to choose.
Configuration files provided with the package use the default
predefined.
With IIS, they use the W3C format (which is really a minimal format), with
Apache, they use the CLF (Common Log File) which have more informations
but still lack the referer, browser and OS informations.
My recommandation would be to change your logfile format to a more
precise one as the ECLF (Extended Common Log File).
If you use your own non-standard logfile format, W3Perl is able to read it
because you can build your own string format using a list of keywords
so there is no limit on logfile format the scripts can read.
Ftp and Squid logfile format have been added recently.
You can have only one
logfile but most servers are splitting the logfiles to avoid the file
becoming too large. W3Perl is able to cope with daily and monthly
split logfiles (they can be compressed also to save disk space).
Log filename should follow some basic rules. They should have a core
filename (i.e access,ex,in ...) and informations about the date. If
you choose to have daily logfiles, the daily date should be in the
log filename, but it can be anything from number to string date.
For the W3C format, it is really necessary as the date is not store
in the log.
News about updated package is available here. If you want to be informed
about the latest release by email, subscribe to the list in the w3perl homepage.
Want to upgrade to the latest W3Perl version ? Please read carefully
those instructions. There are three steps in upgrading.
First is to backup your configuration files just in case ... and any
files you have make some modifications by yourself.
Then run the upgrade script either from command line or from the web
interface. It will update your configuration files if needed.
At last, run the fixperpath script to change the perl path in the new
installed scripts.
Any problem using W3Perl ? Please read first the FAQ. Many questions
have been solved here, including customisation, installation, logfile,
setup and windows issues.
If you don't find your answer there, feel free
to ask in the forum, maybe someone have the answers. You can also
send me an email.
Various hints are available here. If you want to add yours, tell me.
Best advice is to test your configuration on a small scale (with short
logfile or using a starting/ending date for the scripts or with the
lowest precision level). Once everything is fine, go for the complete run.
The major criticism about W3Perl is the speed. The package is written
in Perl so it can't be as fast as C package of course. You could
disable some options to get faster (reverse dns is very slow as it
queries DNS server for each new IP found).
Some tests have been done with a 'slow' computer (right now in 2006)
and it show that W3Perl can process some quite large logfile in a few
hours.
This is the place to ask questions, to submit features and requests
help.
If you want new forum to be added, send me an email. I'll try to
answer everyone as fast as possible.
If you want to be keep inform about the latest release, subscribe to
this notification mailing list. Don't be afraid, you will receive very
few emails a year. This is a easy way for me to announce a new version
available to all W3Perl users.
A form have been installed to keep track of bugs. Please use it to
send problems you have found. Try to be as precise as possible as it
will help me a lot to reproduce the problem.
The list of information I would like to receive is the following :
- W3Perl version
- Name of the script which fail
- How do you install W3Perl (FTP or telnet)
- Your OS
- The step where the problem occur (installation, configuration,
running, web interface...)
- Which kind of logfile are you using (split, compressed, filename)
- The logfile format you are using (CLF,IIS-W3C,IIS-Microsoft...)
- If you have used the web interface to build the configuration
file
- The URL where I can view the problem (if available).
- Your email of course (or I won't be able to contact you back)
- Any comments which could be useful to debug
- Your configuration file and scripts output.
Watch the kind of features I would like to add in the next release. Of
course, this is a wish list. If you have others idea, let me know, I'm
always open to suggestions.
About the display, I would like to add some CSV, PDF or XML output.
About installation, providing more predefined configuration files
would be helpful. Improving the windows installer with the ability
to choose on which server you want to install the package or to
disable the webserver detection to allow installation without IIS or
Apache.
About features, I've planned to add more support for FTP logfiles, to
improve speed and memory usage and to increase the whole package stability.
W3Perl have some limitations as a minimum of one
request by week for the logfile activities or the maximum number of
logfiles to scan.
You'll find the last bug reports and how they have been fixed.
|
