Ramblings of a Tampa developer

I manage a server that runs a handful of websites. I don't have unmetered bandwidth. I have about 1TB of transfer on a 100mbps port. I am supposed to be emailed at 75%, 80%, 90% and 95% usage of bandwidth. In the 2 years I've been doing this, I've never once passed 75%. I have an HTC Inspire with push email, so whenever I'm on standby all communication is forwarded to my phone and I'm able to do any diagnostic, or head back to grab my laptop. What happened next was either a carefully planned attack or a fluke accident. My email related processes (dovecot and exim) somehow both crashed and could not be automatically restarted. In that instant that should trigger a High alert which dispatches emails, text messages and various other methods of communication. I never received anything. Then the true attack started. This attack was a slow but effective DDoS to re-load all the images on my server over and over without caching them. I was loosing GBs of transfer every few minutes.

I had no indication or clue what was going on at this time. A couple of hours later I get a call from my datacenter asking if I was aware that I was using over my 1TB of transfer and that I was being charged .025 per MB (or something close to that). Why a datacenter that profits from my mistake calls me, I will never know, but am eternally grateful. I signed on to check the damage, then I saw my email services were down. They could not be restarted, so I quickly rebuilt them from source and then they loaded up. Then I got the mass load of emails and text messages. Just a couple of hours too late :(

It was a $802 fine for going over my bandwidth limit. I'm now working on something to prevent that from happening again. Some little helpful htaccess scripts have stopped non-referrer and user-agents from viewing directly linked images. That should prevent something like this from happening again.