Example python to perform GeoIP lookups for a list of IP addresses and then generate heatmap using http://www.sethoscope.net/heatmap/
This example walks through downloading and installing the necessary tools to take a list of IP addresses, perform GeoIP lookups to map those to latitude/longitude coordinates, and generate a geographical heatmap using that data.
This procedure leverages several existing libraries and tools:
-
GeoIP database and C/Python libraries to query the database
-
Python libraries to work with OpenStreetMap data and images
-
Python script that uses the OpenStreetMap/Imaging libraries to generate a heatmap to visualize a set latitude/longitude coordinates
Using the above, I wrote a quick Python script to perform GeoIP lookups for all IP addresses listed in an input file that we can then feed into the heatmap script to generate our map.
This document will walk you through building all of the prerequisite tools in your home directory.
These instructions will walk you through the following:
-
Download latest GeoIP database
-
Download and install C/Python libraries to work with local GeoIP database
-
Download prerequisites for heatmap Python script
-
Generate list of recent IP addresses
-
Convert list of recent IP addresses into their latitude/longitude coordinates based on GeoIP database
-
Generate heatmap image using this location data
$ wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
$ gunzip GeoLiteCity.dat.gz
$ cd $HOME/geoip-heatmap-python
$ wget https://www.maxmind.com/download/geoip/api/c/GeoIP-1.4.8.tar.gz
$ tar -zxf GeoIP-1.4.8.tar.gz
$ cd GeoIP-1.4.8
$ ./configure --prefix=$HOME/geoip-heatmap-python/build
$ make
$ make install
$ export PATH=$HOME/geoip-heatmap-python/build/bin:$PATH
$ export LD_LIBRARY_PATH=$HOME/geoip-heatmap-python/build/lib:$LD_LIBRARY_PATH
$ cd $HOME/geoip-heatmap-python
$ wget https://www.maxmind.com/download/geoip/api/python/GeoIP-Python-1.2.7.tar.gz
$ tar -zxf GeoIP-Python-1.2.7.tar.gz
$ cd GeoIP-Python-1.2.7
Modify library_dirs and include_dirs variables in setup.py to include our GeoIP C library install directory
$ vim setup.py
$ python setup.py build
$ python setup.py install --prefix=$HOME/geoip-heatmap-python/build
$ export PYTHONPATH=$HOME/geoip-heatmap-python/build/lib64/python2.6/site-packages/:$PYTHONPATH
$ cd $HOME/geoip-heatmap-python
$ wget https://github.com/cbick/osmviz/archive/master.tar.gz
$ tar -zxf osmviz-master.tar.gz
$ cp -R osmviz-master/src/* $HOME/geoip-heatmap-python/build/lib64/python2.6/site-packages/
$ cd $HOME/geoip-heatmap-python
$ wget http://effbot.org/downloads/Imaging-1.1.7.tar.gz
$ tar -zxf Imaging-1.1.7.tar.gz
$ cd Imaging-1.1.7/
$ python setup.py install --prefix=$HOME/geoip-heatmap-python/build
$ cd $HOME/geoip-heatmap-python
$ last -i | awk '{print $3}' | grep ^[0-9] > recent-ips.txt
Convert list of recent IP addresses into their latitude/longitude coordinates based on GeoIP database
$ cd $HOME/geoip-heatmap-python
$ python geolite.py recent-ips.txt > recent-ips.points
Get the heatmap python script from http://www.sethoscope.net/heatmap/
$ cd $HOME/geoip-heatmap-python
$ wget http://www.sethoscope.net/heatmap/heatmap.py
$ python heatmap.py -p recent-ips.points -o recent-ips.png --width 800 --osm -d 0.5