Simple solution to protect your Drupal website/VPS/webserver from DDOS by crawler bots that cause apache load spikes
https://www.zyxware.com/sites/default/files/styles/user_image/public/default_images/index.png?itok=2YmREnrP
BY Anoop John
7 years ago
Drupal
0 comments comment

If you run a reasonably popular site there is a very good chance that you will get crawled by unscrupulous crawlers once in a while. The regular search crawlers from the popular search engines like the Google bot, Yahoo bot, Bing bot themselves are pretty intensive when they crawl sites but they do have limits on the number of pages crawled per visit and the number of simultaneous connections opened to your server. However there are lots of unscrupulous crawlers like spam bots, email harvesting bots and even some search engines who do not set any reasonable limits on the number of simultaneous connections to your server. There are hardware solutions that are available to prevent this but they may not be affordable for everybody. There is however a simple netstat based solution to prevent DDOS.

There is this small application called DDOS-deflate that uses a netstat based approach to ban IPs that opens too many connections to your server. The script sets up a cron job which can be run as frequently as every minute to run ban such IPs.

The installation is very simple. You can run the following commands to install DDOS deflate on your server

wget http://www.inetbase.com/scripts/ddos/install.sh
chmod 0700 install.sh
./install.sh

The application gets installed to /usr/local/ddos. There is a simple configuration file located at /usr/local/ddos/ddos.conf where you can configure the application and set the threshold on the number of connections above which you should ban an IP and the seconds for which the IP should be banned.

The application runs the following command

netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n

to catch the problem IPs and then ban them either using AFP if your system has it installed or using iptables.

If you are adventurous enough you can disable the DDOS-deflate cron job and write your own shell script that runs in the background and calls the DDOS-deflate script more frequenly than once per minute as cron jobs have a minimum resolution of 1 minute. If you are running a Drupal site then the impact of the DDOS attack would be much higher than a static website. You would also want to ensure that you have taken care of the minimal performance configurations for Drupal.


RELATED ARTICLE

/themes/custom/zyxpro_light/images/placeholder.png
close

on 01st January 2008 / by webmaster
We have added a few new modules to our site recently. Open Source Web Development Drupal Blogging Free Software Drupalgive Leave a reply Your email address will not be published. Required fields are marked * Sean (not verified) access_time 13 Sep 2019 - 20:23 Hi there, I uploaded the files, enabled the modules to use SU, and now I see the link, but there is no icon... How can I fix this? Thanks, Sean webmaster access_time 13 Sep 2019 - 20:23 In reply to Icon doesn't show by Sean (not verified) What do you see when you view the source? Search for stumblethis_button and you should be able to see the code for the image and the URL. Then you should be able to troubleshoot from there. Juicy Couture Addict (not verified) access_time 13 Sep 2019 - 20:23 thanks for the post. would love to hear more of you. by the way, drupal's really popular nowadays as it has easy and fast features. you agree with me? thanks. Add new comment
/themes/custom/zyxpro_light/images/placeholder.png
more_horiz
close

on 16th January 2008 / by webmaster
Most webmasters do not realize this, but a lot of the content on lot of websites can be accessed from multiple URLs. A simple example would be where www.example.com and example.com leads to the same page. This is a fatal mistake in Search Engine Optimization and search engines penalize you for duplicate content. The correct configuration would be where the above two urls will lead you to the same page but example.com will redirect you to www.example.com with a 301 (Moved permanently) status which will not result in search engines penalizing the page. It is very easy to configure 301 redirects using Apache .htaccess file and the process is the same for a Drupal installation also. Web Development Drupal SEO Drupal Planet Leave a reply Your email address will not be published. Required fields are marked * Anonymous (not verified) access_time 13 Sep 2019 - 20:23 Hello. I'm trying to make example.com show as www.example.com, and I'm running into difficulties. I'm on Apache 2.0 and using the following lines in my httpd.conf file: RewriteEngine on RewriteCond %{HTTP_HOST} ^xxxxxxxxxx\.com$ [NC] RewriteRule ^(.*)$ http://www.xxxxxxxxxx.com/$1 [L,R=301] When I go to http://xxxxxxxxxx.com, I get http://www.xxxxxxxxxx.com (as expected). However, when I go to http://xxxxxxxxxx.com/node/1 (it's a Drupal site), I get a 404 thrown and the URL changes to xxxxxxxxxx.com/var/www/drupal/node/1. Same thing with www.xxxxxxxxxx.com/node/1. Any suggestions? I want to run without Drupal's .htaccess file (instead incorporating these calls into my httpd.conf file). webmaster access_time 13 Sep 2019 - 20:23 In reply to Rewrite including filesystem path by Anonymous (not verified) I think the problem is with the base path which results in the redirection to /var/www/ part. The best approach I would think is to start with drupal htaccess and then strip out parts and move to httpd SNVC (not verified) access_time 13 Sep 2019 - 20:23 This is definitely a good guide. Thanks for this. wellyson access_time 13 Sep 2019 - 20:23 This is really nice and helpful. Add new comment
/themes/custom/zyxpro_light/images/placeholder.png
close

on 07th January 2008 / by webmaster
We have volunteered to take up the maintenance of a very useful Drupal module - Search404. As of today we are the official maintainers of this very useful Drupal module. We know that this is going to be a challenge for us, being a young company and with a young team. But we do feel that it is our responsibility to give back to the Drupal community at least some part of what it has given us. Open Source Web Development Drupal Leave a reply Your email address will not be published. Required fields are marked * ian douglas (not verified) access_time 13 Sep 2019 - 20:23 I notice your updates on the Drupal modules page has some patch files, but they are patches for an older version of the 5.x branch of search404. Do you have any expectation on when a version for Drupal 6 will be ready? webmaster access_time 13 Sep 2019 - 20:23 In reply to when will search404 for drupal 6 be ready? by ian douglas (not verified) Hi Ian, We are currently working on moving Search404 to Drupal 6. The port has already been done and we should have a release up on drupal.org by tomorrow or worst case by monday. Cheers Anoop John Team Zyxware Dejan (not verified) access_time 13 Sep 2019 - 20:23 In reply to when will search404 for drupal 6 be ready? by ian douglas (not verified) I think that's been up for a while... did you check the download page? Add new comment
Leave a reply
Your email address will not be published. Required fields are marked *

Filtered HTML

  • Web page addresses and email addresses turn into links automatically.
  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type='1 A I'> <li> <dl> <dt> <dd> <h2 id='jump-*'> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
The content of this field is kept private and will not be shown publicly.
CAPTCHA This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.