IP Clustering with Linux

These are very preliminary instructions for how to add IP clustering support to Linux. It's basically a copy of an email I sent which is itself just a brain dump from memory. I'll update this properly when I get a chance.

This all assumes kernel 2.0.32. It's not yet ported to 2.1. First, get the current distribution. Note that the directory is executable but unreadable so if you use another ftp client you'll have to give the exact name of the file.

Drop ip_cluster.c into net/ipv4, replace ip_{fw,input}.c with the new versions, replace necessary files in include/linux and include/net with linux/*.h and net/*.h from the ip_cluster dist. Apply the following patch:

--- net/ipv4/Config.in.ORI      Mon Dec  1 12:59:27 1997
+++ net/ipv4/Config.in  Mon Dec  1 13:03:00 1997
@@ -21,6 +21,7 @@
       fi
       if [ "$CONFIG_EXPERIMENTAL" = "y" ]; then
         bool 'IP: transparent proxy support (EXPERIMENTAL)' CONFIG_IP_TRANSPARENT_PROXY
+        bool 'IP: clustering support' CONFIG_IP_CLUSTER
       fi
      bool 'IP: always defragment' CONFIG_IP_ALWAYS_DEFRAG
   fi
--- net/ipv4/Makefile.ORI       Mon Dec  1 13:03:42 1997
+++ net/ipv4/Makefile   Mon Dec  1 13:04:49 1997
@@ -44,6 +44,10 @@
 M_OBJS += ip_masq_vdolive.o ip_masq_quake.o
 endif

+ifeq ($(CONFIG_IP_CLUSTER),y)
+IPV4_OBJS += ip_cluster.o
+endif
+
 ifeq ($(CONFIG_IP_ALIAS),y)
 IPV4_OBJS += ip_alias.o
 else
Configure the kernel and turn on firewalling, clustering and plenty of other similar network stuff (I can't remember off-hand exactly what CONFIG_IP_CLUSTER relies on). Build a kernel from that to run on the front-end (other nodes can use normal kernels: they only need IP alias). Do a "make" in the ip_cluster directory to build the pool module, the clst_debug module and clconfig (user-mode cluster config program). Compile the hacked ipfwadm in there (not in Makefile yet). Basically, the cluster nodes should be in a "private" network (e.g. 10.0.0.0) and all use the front-end node as their default gateway. The front-end should be the only link the the outside world. ifconfig each cluster node with its own private address (with a direct net route to the rest of the private net). For the "shared" IP address, do an "ifconfig eth0:0 1.2.3.4 -arp" on each node. On the front-end do insmod pool to load a pool device (the first will be called pool0). Then do
    clconfig -add pool0 10.0.0.1 up cost 10
    clconfig -add pool0 10.0.0.2 up cost 10
for each cluster node. Do an ifconfig eth0:0 1.2.3.4 on the front-end so that it sees traffic to the shared IP address. Then add a firewall rule on the front-end something like
    ipfwadm -I -b -a cluster -S 0/0 -D 1.2.3.4/32 -W pool0
which will make those packets go to pool0. New connections will be redirected to the least loaded node (you can use -V 10.0.0.5) if you want to redirest to a static cluster node). You can do
    cat /proc/net/ip_cluster
to see the current state of clustered connections. All the above is off the top of my head so there may be errors.


Back to home page