
One of the challenges with working with dynamic websites is that you have to keep fighting malicious users who regularly sap your server capacity with rogue crawling of your site. To do this you would have to monitor and analyze the traffic patterns on the server regularly. You would definitely want to do this when you have load spikes on the server and you wish to find out the IP addresses, user agents and the specific URLs which resulted in these spikes.