NetMon

March 17 2015

These are my notes on building the IP/location lookup tables for NetMon. This is the static database that lets NetMon look up the geographic location of an IP address.

The database is provided by MaxMind. I am using the GeoLite2 City database, which is provided under the following distribution model:

The GeoLite2 databases are distributed under the Creative Commons Attribution-ShareAlike 3.0 Unported License. The attribution requirement may be met by including the following in all advertising and documentation mentioning features of or use of this database:

This product includes GeoLite2 data created by MaxMind, available from http://www.maxmind.com.

Updating the GeoDB Database

1. Download the latest source database
   GeoLite2-City-CSV.zip
2. Unpack the source database
   Save the original zipfile to NetMon\Data\Zip\
   Delete the previous GeoDB output, everything in GeoDB\Data\Test\
   Unzip the contents to NetMon\Data\Cur\
3. Convert the source database to GeoDB
   GeoDB> ..\Out\Winx64Release\GeoDBcreate --create --in Data\Cur --out Data\Test --lookup 109.255.200.0
   Verify that the lookup succeeds and finds Irishtown.
4. Update GeoDB
   Copy everything in GeoDB\Data\Test\ to the NetMon installation directory. For development, this is NetMon\Test\.

These steps have been automated in the GeoDB_Update.bat script.

March 20 2015

The core GeoDB functions have been extracted to the project GeoLib which produces GeoDB.lib. The database converter program has been renamed to GeoDBcreate.exe, although the project is still named GeoDB.

March 19 2015

The update to using GeoLite2 affected most of GeoDB. Not a complete rewrite but definitely major surgery. Every rewrite is an evolution.

I added the ability to perform a single lookup after the database is created. This lets me quickly test the database without relying on NetMon.sy and NetMonUI.exe. The test IP is 80.133.68.0/24, which maps to geoname_id
9972705 EU,Europe,DE,Germany,NI,"Lower Saxony",,,Friedland,,Europe/Berlin
which is the third-penultimate row in the locations table.

An even more interesting IP is 109.255.200.0/25:
geoname_id 9866845 locale_code en continent_code EU continent_name Europe country_iso_code IE country_name Ireland subdivision_1_iso_code L subdivision_1_name Leinster subdivision_2_iso_code MH subdivision_2_name "Co Meath" city_name Irishtown metro_code time_zone Europe/Dublin

I finished the basic lookup code, fixed a couple parsing bugs, then cleaned up the code. This is in preparation for moving it into a GeoDB library where it can be shared with GeoDB.exe and NetMonUI.exe. I went the extra mile and wrote a binary search. The possibility that the IpIndex.db (nee Octet.db) entries could theoretically span a very large number of IP blocks always bothered me. My test case, just a random IP address, proved the point:

So even when the target is only 62/233 entries off from the IpIndex, the binary search still finds the target in far fewer iterations. I do not know, however, the impact of randomly reading records from disk versus reading them linearly. I still expect the binary search to be faster.