Using Weathermap with Cacti

Created: Jan 10th 2005
Updated: Jan 22nd 2005 - v1.1.1hj2
Updated: Jan 28th 2005 - v1.1.1hj3
(Final version - This has been replaced by PHP Weathermap and it is no longer in development)

(not finished! but I thought it was useful enough to put up somewhere. Sorry about the wide page, but some of the lines are wide!). You might also want to look into the more recent part of this thread on the Cacti forum.

Cette page est également disponible en français, grâce au travail de Jérémy Couture.

Sample Map

A network map generated by this script, using a custom background image. The link arrows change colour with the utilisation of the line. Data is read from RRDs created by Cacti.
Larger Version.

Weathermap is a perl script designed to take data from an MRTG installation, and present it like the chart shown here. Cacti is a php-based replacement for MRTG, with an integrated front-end and extensive templating system. This page explains some changes for Weathermap to get it to play friendly with Cacti, and also to produce the same style of map shown on Weathermap's demo page, without as much manual work as previously was required.

Note: This page includes a modified version of Panagiotis Christias Weathermap script. I did try and contact the author about integrating the patches, but recieved no reply. Since I'd already mentioned that I had these changes on the Cacti forums, I thought I'd put them up here. The original script is licensed under the GPL, so that's OK to do, although I'd rather not end up forking the original if possible.

Introduction

I'm assuming here that you already have Cacti working, and are happy with it. You should follow through the docs that came with Cacti first. This weathermap setup doesn't require any changes to your Cacti configuration to work, although there is one small change that will make it all a bit neater.

Once you have Cacti working...

cacti2weathermap.pl

IMPORTANT As of 1.1.1hj3, you no longer need cacti2weathermap.pl - you can specify the RRD files from cacti as a TARGET directly.

rama on the Cacti forum has posted the script he uses to get weathermap-compatible data out of cacti. Basically, weathermap relies on some HTML comments that MRTG would normally put at the bottom of it's HTML pages, which contain the last-measured bandwidth statistics. rama's script looks into Cacti's RRD files and produces some minimal HTML files with the same style of comments that MRTG would produce.

To use it, I created a new directory inside my base cacti directory called weathermap, and another inside that called data and chowned them both to the cactiuser, so that cacti itself can write data to it.

cd /usr/local/www/data/cacti (wherever you installed cacti)
mkdir weathermap
mkdir weathermap/data
chown -R cactiuser weathermap (or whatever user 'owns' your cacti)

I put the script itself in the weathermap directory, then changed the two variables at the top to match my installation. Here's my modified copy (it allows to specify the source directory on the command-line, and has a couple of error checks):

#!/usr/bin/perl

# you might need to uncomment this line, or one like it
# use lib qw(/usr/local/rrdtool-1.0.49/lib/perl);
use RRDs;

#Path to rrd files, or take a command-line option if it exists
$my_dir = $ARGV[0] || "/Users/howie/Sites/weathermap/cacti/rra/";

#Path to store weathermap compatible files
$my_htmldir = "data/";

die("RRA directory specified ($my_dir) doesn't exist.") if ! -d $my_dir;
die("Output directory ($my_html_dir) doesn't exist.") if ! -d $my_htmldir;

opendir RRDDIR, $my_dir or die "Impossible to open file: $!";
@rrd_files = grep /rrd/, readdir RRDDIR;

foreach my $router (@rrd_files){
        print $router,"\n";
        #Get the last cuin/cuout values
        my ($start,$step,$names,$data) = RRDs::fetch $my_dir.$router,"AVERAGE","--start","now";
        print "$start,$names  \n";
        foreach my $line (@$data) {
            if( defined(@$line[0]) ) {
                (my $name, my $ext)  = split /rrd/,$router;
                $weatherfile = $my_htmldir."wea_".$name."html";
                print $weatherfile,"\n";
                #Open special html weathermap file
                open OUT,"%gt;$weatherfile" or die "bad luck: $!";
                print OUT "%lt;html%gt;\n%lt;body%gt;\n$router\n";
                $nice = sprintf "%lt;!-- cuin d %d --%gt;\n%lt;!-- cuout d %d --%gt;\n", @$line[0], @$line[1];
                print OUT $nice;
                print OUT "%lt;/body%gt;\n%lt;/html%gt;";
                close OUT;
                }
        }
}

That done, you should be able to run the script and see a lot of small (100 byte) html files appear in the weathermap/data directory.

cd /usr/local/www/data/cacti/weathermap/
./cacti2weathermap.pl
ls -l data
-rw-r--r--  1 cactiuser  howie    99 Jan 10 14:18 wea_xlr_traffic_in_345.html
-rw-r--r--  1 cactiuser  howie   103 Jan 10 14:18 wea_xlr_traffic_in_346.html
-rw-r--r--  1 cactiuser  howie    93 Jan 10 14:18 wea_xlr_traffic_in_439.html
-rw-r--r--  1 cactiuser  howie    95 Jan 10 14:18 wea_xlr_traffic_in_441.html
-rw-r--r--  1 cactiuser  howie    97 Jan 10 14:18 wea_xlr_unicast_in_347.html
[etc etc]

weathermap.pl changes

Now you've got the required HTML files, you can install WeatherMap and tell it where to find the information it needs. The actual network topology used by WeatherMap is defined in a text config file, along with the maximum bandwidth of the line, and where to get it's data from.

This version has one more perl module requirement than the original: Number::Format is used to make the bandwidth figures format nicely. i've extended the original WeatherMap configuration file format with a few new keywords. If you don't use them, then you get the same results as with the standard program (aside from one difference - the bandwidth labels use a smaller font). The new keywords are (see the excellent docs on the original weathermap page for the basics):

HTMLSTYLE overlib

(global command) Rather than a static HTML page, generate one with the necessary JavaScript to make a map with popup graphs like the one linked to from the WeatherMap home page.

BWLABELS bytes

(global command) Rather than the default percentages, show the current actual bandwidth in the middle of the arrow for each link. What we found was that with lots of 100Mbit/sec links, the percentages tend to show 1% a lot, whereas we'd rather know the difference between 1bit/sec and 1Mbit/sec. The colour is still calculated from the percentage.

OVERLIBGRAPH

(per LINK or NODE) If you use the --html-output command line option (see below) and the 'HTMLSTYLE overlib' config option (see above), then this is the URL for an image that will pop up when the user hovers the mouse over the item. Intended to be the '5-minute' graph from MRTG or Cacti. For Cacti it'll be something like:

http://www.server.tld/cacti/graph_image.php?local_graph_id=343&
rra_id=0&graph_nolegend=true&graph_height=100&graph_width=300

INFOURL

(per LINK or NODE) If you click on the link arrow or node, this is the URL to go to. Intended to be the URL of the cacti/MRTG page with the full history for the object. For Cacti, it'll be something like:

http://www.server.tld/cacti/graph.php?rra_id=all&local_graph_id=343

CHANGEDIRECTION

(global setting) If you have a lot of maps already, and you were using cacti2weathermap.pl previously, and want to switch to reading RRDs directly, then you'll need to edit your map files fairly extensively to swap the order of the node names for each LINK. This is because cacti2weathermap.pl actually pulls the data out in the wrong order. Alternatively, you can use this option to have weathermap swap ALL in/out values, saving you the trouble. Values can be 0 (default) or 1 (do the swap).

You can use the INFOURL without the OVERLIBGRAPH, or choose to only have some links use either option. Without them, you'll get the default behaviour.

Also new is a command-line option: --html-output filename
This asks weathermap.pl to generate an HTML file as well as the PNG file it usually produces. If you choose the 'HTMLSTYLE overlib' option in the config file, then this HTML file will pull in the overlib_mini.js file needed to make the popup graphs.

Since that produces another wrinkle, there is also another command-line option: --image-uri URI
By default weathermap will use the exact name given to --output as the SRC in the IMG tag, if you ask it to generate HTML. If you don't specify --output then it defaults to weathermap.png, which will work OK, but if you DO, and it's a full path, it's probably not a valid URI. This lets you specify the URI manually. For example:

./weathermap --html-output weathermap.html --output /var/www/cacti/weathermap/weathermap.png  \
		--image-uri /weathermap/weathermap.png --config weathermap.conf

Finally, as another small but useful tweak. This version of WeatherMap puts a timestamp in the top corner, so that you can tell if the map is being updated at all!

Some Assembly Required

(TODO: how to find the right graph and RRA numbers in cacti...)

Once you have something that seems right, you should be able to run

./weathermap --html-output weathermap.html --output weathermap.png  --config weathermap.conf

and get a sensible looking map. This is the last step in the chain.

Since I've been bitten by it a couple of times so far, it's worth mentioning that you should be extra careful about the order of the node names when defining a link. When you have a link defined like

LINK lon-ams
        NODES amsterdam london
        BANDWIDTH 1000000
        TARGET data/wea_cisco1_traffic_in_352.html
        OVERLIBGRAPH http://cacti.company.net/cacti/graph_image.php?local_graph_id=436&rra_id=0&graph_nolegend=true&graph_height=100&graph_width=300 
        INFOURL http://cacti.company.net/cacti/graph.php?rra_id=all&local_graph_id=436

Make sure that the information shown in the map is in the right direction! You might need to swap the two names to get the right view of your network, if you are seeing a lot of data apparently flowing into strange corners.

Altogether now!

The easiest way to get this all to work together nicely is to run it all from the same crontab entry as you already run cacti from. To do this, make a shell-script in your Cacti base directory that looks something like:

#!/bin/sh

#change the next line to point to the base of your cacti installation
CACTIBASE="/usr/local/www/data/cacti"
WMAPBASE="$CACTIBASE/weathermap"

# do the normal cacti data update
/usr/local/bin/php $CACTIBASE/poller.php 

cd $WMAPBASE

# make the HTML files that weathermap needs
$WMAPBASE/cacti2weathermap.pl $CACTIBASE/rra/

# make the weathermaps
$WMAPBASE/weathermap --htmloutput $WMAPBASE/weathermap-live.html --config $WMAPBASE/weathermap.conf --output weathermap.png

Then change the crontab entry you already have to run poller.php to run update.sh (or whatever you called this shellscript) instead. This way, cacti will have definitely finished polling for new data before the map is generated, guaranteeing the latest information.

Downloads

weathermap-1.1.1hj3.tar.gz(127Kb): My altered version of weathermap. This includes the changes documented above, a modified version of cacti2weathermap.pl and the update.sh shellscript. Unpack the tar into your cacti base directory to get a new subdirectory called weathermap, and (hopefully) all the paths set up right. You should just need to change the top of update.sh
weathermap-1.1.1hj3.exe(2370Kb): A Windows executable made using PAR, that should be a self-contained version of the above script. You'll need the tar too though, as this is just the executable, with no docs or examples. This is really just an experiment, but it does work for me.

ChangeLog

v1.1.1hj3: 22 Jan 2005 (I was bored!)
        Added direct reading of RRDs. Just use the RRD file as the TARGET
        Added option to allow swapping of in/out (cacti2weathermap.html gets
                in and out the wrong way round from RRDs! REVERSEDIRECTION
                allows you to use 'old' configs with the current code without
                having to re-do them.

v1.1.1hj2: 21 Jan 2005
	Added --base-directory command-line option.
	Added OVERLIBGRAPH and INFOURL for nodes.
	Added WIDTH option for LINKs. Default is 7 as previously.
	Updated --help output
	Fixed up HTML output a bit - no do-nothing imagemap areas anymore.

v1.1.1hj: 19 Jan 2005
	Initial public release.
	Adds OVERLIBGRAPH and INFOURL config lines for LINKs
	Adds HTML and imagemap generation
	Adds HTMLSTYLE config option
	Adds BWLABELS config option
	Adds map timestamp

v1.1.1: 3 Apr 2004
	p.christias@noc.ntua.gr version that I started with.