CIDRMerge

For the impatients: Link to sourceforge download page

Introduction

The program implements the CIDR merging with white listing (network exclusion). Output will be summarized to be as small as possible.

The program has been widely used for blacklist merge and optimization: you can put together many blacklist, apply your own white list and obtain the optimized version of input files. It works perfectly in conjunction with RBLDNS. Many big ISP are currently using it.

It can be used under certain conditions for reducing number of lines of ACLs or routing tables. It can be use to compute network complement too.

Performances

The algorithm in both optimization cases (with and without white list) is executed with the N*log(N) complexity due to sorting function.

The sort is realized using QSORT function supplied by Michael Tokarev.

Memory usage is not an issue: with an about 9million networks blacklist it uses between 50M to 80M RAM (depending on compile time options and machine capability).

Usage
cidrmerge -h|-v|[whitelist file] [NOOPTIMIZE]
-v              Print version number and exit
-h              Print small help hints and exit
[whitelist file] CIDR format file containing whitelist
[NOOPTIMIZE] will apply the whitelist but will not cidrmerge the output. Useful for debug purposes

Return code: 0 on success, > 0 in case of error

Optimization Example

A (Taken from standard input)
10.1.1.0/24
10.1.1.1/32
192.1.4.5/32
192.1.4.4/32
B
10.1.1.0/24
192.1.4.3/31

White list Example
A (Taken from standard input)
10.1.1.0/24
10.1.1.1/32
192.1.4.5/32
192.1.4.4/32
B (read as input file)
192.1.4.4/32
10.1.1.128/31
C
10.1.1.0/25
10.1.1.130/31
10.1.1.132/30
10.1.1.136/29
10.1.1.144/28
10.1.1.160/27
10.1.1.192/26
10.1.1.1/32
192.1.4.5/32

Project Components

This programs consist of some part:

1) the optimization library: The library offer core functionality that can be used by programmers to build their own programs.It can be used as shared library or statically linked

2) the optimization program: an implementaton that read CIDR formatted lines and output CIDR formatted lines, using core library function accordingly

3) two utility script: one to convert network form addresses (network/netmask) to CIDR called netmasktocidr.pl and the opposite, called cidrtonetmask.pl

Build

A C compiler is needed (tested with gcc 2.95, 3.2 and 3.3). Simply type make will work in most cases.

There are some tunable paramethers into Makefile, please have a look to it if you want to have top performances.

To compile shared library, please use "make lib".

Academic degree thesis

I've analized the algorithm and the progam in depth. Sorry, in italian :-(

AUTHOR

Daniele Depetrini (depetrini[{At}]libero.it)

from http://cidrmerge.sourceforge.net/