Simple solution to protect your Drupal website/VPS/webserver from DDOS by crawler bots that cause apache load spikes

If you run a reasonably popular site there is a very good chance that you will get crawled by unscrupulous crawlers once in a while. The regular search crawlers from the popular search engines like the Google bot, Yahoo bot, Bing bot themselves are pretty intensive when they crawl sites but they do have limits on the number of pages crawled per visit and the number of simultaneous connections opened to your server. However there are lots of unscrupulous crawlers like spam bots, email harvesting bots and even some search engines who do not set any reasonable limits on the number of simultaneous connections to your server. There are hardware solutions that are available to prevent this but they may not be affordable for everybody. There is however a simple netstat based solution to prevent DDOS.

There is this small application called DDOS-deflate that uses a netstat based approach to ban IPs that opens too many connections to your server. The script sets up a cron job which can be run as frequently as every minute to run ban such IPs.

The installation is very simple. You can run the following commands to install DDOS deflate on your server

wget http://www.inetbase.com/scripts/ddos/install.sh
chmod 0700 install.sh
./install.sh

The application gets installed to /usr/local/ddos. There is a simple configuration file located at /usr/local/ddos/ddos.conf where you can configure the application and set the threshold on the number of connections above which you should ban an IP and the seconds for which the IP should be banned.

The application runs the following command

netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n

to catch the problem IPs and then ban them either using AFP if your system has it installed or using iptables.

If you are adventurous enough you can disable the DDOS-deflate cron job and write your own shell script that runs in the background and calls the DDOS-deflate script more frequenly than once per minute as cron jobs have a minimum resolution of 1 minute. If you are running a Drupal site then the impact of the DDOS attack would be much higher than a static website. You would also want to ensure that you have taken care of the minimal performance configurations for Drupal.