Simple solution to protect your Drupal website/VPS/webserver from DDOS by crawler bots that cause apache load spikes
https://www.zyxware.com/sites/default/files/styles/user_image/public/default_images/index.png?itok=2YmREnrP
BY Anoop John
8 years ago
Drupal-Technical
0 comments comment

If you run a reasonably popular site there is a very good chance that you will get crawled by unscrupulous crawlers once in a while. The regular search crawlers from the popular search engines like the Google bot, Yahoo bot, Bing bot themselves are pretty intensive when they crawl sites but they do have limits on the number of pages crawled per visit and the number of simultaneous connections opened to your server. However there are lots of unscrupulous crawlers like spam bots, email harvesting bots and even some search engines who do not set any reasonable limits on the number of simultaneous connections to your server. There are hardware solutions that are available to prevent this but they may not be affordable for everybody. There is however a simple netstat based solution to prevent DDOS.

There is this small application called DDOS-deflate that uses a netstat based approach to ban IPs that opens too many connections to your server. The script sets up a cron job which can be run as frequently as every minute to run ban such IPs.

The installation is very simple. You can run the following commands to install DDOS deflate on your server

wget http://www.inetbase.com/scripts/ddos/install.sh
chmod 0700 install.sh
./install.sh

The application gets installed to /usr/local/ddos. There is a simple configuration file located at /usr/local/ddos/ddos.conf where you can configure the application and set the threshold on the number of connections above which you should ban an IP and the seconds for which the IP should be banned.

The application runs the following command

netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n

to catch the problem IPs and then ban them either using AFP if your system has it installed or using iptables.

If you are adventurous enough you can disable the DDOS-deflate cron job and write your own shell script that runs in the background and calls the DDOS-deflate script more frequenly than once per minute as cron jobs have a minimum resolution of 1 minute. If you are running a Drupal site then the impact of the DDOS attack would be much higher than a static website. You would also want to ensure that you have taken care of the minimal performance configurations for Drupal.


RELATED ARTICLE

/themes/custom/zyxpro_light/images/placeholder.png
close

on 01st January 2008 / by webmaster
We have added a few new modules to our site recently. Drupal Technical StumbleThis Leave a reply Your email address will not be published. Required fields are marked * Sean (not verified) access_time 11 Aug 2020 - 11:38 Hi there, I uploaded the files, enabled the modules to use SU, and now I see the link, but there is no icon... How can I fix this? Thanks, Sean webmaster access_time 11 Aug 2020 - 11:38 In reply to Icon doesn't show by Sean (not verified) What do you see when you view the source? Search for stumblethis_button and you should be able to see the code for the image and the URL. Then you should be able to troubleshoot from there. Juicy Couture Addict (not verified) access_time 11 Aug 2020 - 11:38 thanks for the post. would love to hear more of you. by the way, drupal's really popular nowadays as it has easy and fast features. you agree with me? thanks. Add new comment
/themes/custom/zyxpro_light/images/placeholder.png
close

on 16th January 2008 / by webmaster
Most webmasters do not realize this, but a lot of the content on lot of websites can be accessed from multiple URLs. A simple example would be where www.example.com and example.com leads to the same page. This is a fatal mistake in Search Engine Optimization and search engines penalize you for duplicate content. The correct configuration would be where the above two urls will lead you to the same page but example.com will redirect you to www.example.com with a 301 (Moved permanently) status which will not result in search engines penalizing the page. It is very easy to configure 301 redirects using Apache .htaccess file and the process is the same for a Drupal installation also. Drupal Technical Apache htaccess SEO Leave a reply Your email address will not be published. Required fields are marked * Anonymous (not verified) access_time 11 Aug 2020 - 11:44 Hello. I'm trying to make example.com show as www.example.com, and I'm running into difficulties. I'm on Apache 2.0 and using the following lines in my httpd.conf file: RewriteEngine on RewriteCond %{HTTP_HOST} ^xxxxxxxxxx\.com$ [NC] RewriteRule ^(.*)$ http://www.xxxxxxxxxx.com/$1 [L,R=301] When I go to http://xxxxxxxxxx.com, I get http://www.xxxxxxxxxx.com (as expected). However, when I go to http://xxxxxxxxxx.com/node/1 (it's a Drupal site), I get a 404 thrown and the URL changes to xxxxxxxxxx.com/var/www/drupal/node/1. Same thing with www.xxxxxxxxxx.com/node/1. Any suggestions? I want to run without Drupal's .htaccess file (instead incorporating these calls into my httpd.conf file). webmaster access_time 11 Aug 2020 - 11:44 In reply to Rewrite including filesystem path by Anonymous (not verified) I think the problem is with the base path which results in the redirection to /var/www/ part. The best approach I would think is to start with drupal htaccess and then strip out parts and move to httpd SNVC (not verified) access_time 11 Aug 2020 - 11:44 This is definitely a good guide. Thanks for this. wellyson access_time 11 Aug 2020 - 11:44 This is really nice and helpful. Add new comment
/themes/custom/zyxpro_light/images/placeholder.png
close

on 08th February 2008 / by webmaster
Attachment Size slashdotit.tar.gz 173.55 KB Similar to the StumbleThis module that we had created earlier we have created another bookmarking Drupal Technical Slashdotit Leave a reply Your email address will not be published. Required fields are marked * Shlomi Fish (not verified) access_time 11 Aug 2020 - 11:38 Hi! I'm interested in this module, but Drupal 6.2 complains that it's incompatible with it. Can you please update it or tell me how? Regards, Shlomi Fish shlomifish.org webmaster access_time 11 Aug 2020 - 11:38 In reply to Can you update it for Drupal-6.x? by Shlomi Fish (not verified) Sure we are working on this currently and will post an update soon. Also we will post this module on www.drupal.org as well Cheers Anoop John Team Zyxware Anonymous (not verified) access_time 11 Aug 2020 - 11:38 Hello! Nice Article very interesting, thanks Add new comment
Leave a reply
Your email address will not be published. Required fields are marked *

Filtered HTML

  • Web page addresses and email addresses turn into links automatically.
  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type='1 A I'> <li> <dl> <dt> <dd> <h2 id='jump-*'> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
The content of this field is kept private and will not be shown publicly.
CAPTCHA This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.