DDoS Protection

[Blackup]

Although Our IPB site hasn't been ddosed yet, it would not surprise me if this would happen in the near future. I'm going to try and stay ahead of this and get ddos protection. Would anyone have any good suggestions?

[Biker.GA]

Not much you can do about it other than running your site across several servers. It's more in the hands of your ISP, and how they're set up to handle such things.

[*Salvo*]

..a good firewall is the unique way to face the problem.
Some silly guy will always enjoy to download a freeware and create a bunch of queries in the same instant. Best thing is detect the IP and ban it from server, and while it happens close server as the entire machine will be affected. That's what I did once. The custom assistance gave me this (good) suggestion. The affected software was Wordpress, BTW.

[.Immortal]

Well, coming from a person who dealt with a couple ddos, depending on how large and how complex the attack is, will depend on how you deal with them. It would also depend on what kind of hosting you have. People who have Shared hosting can do very little compared to say someone on a dedicated server due to the person having more control and able to implement protection against it.

People who have shared hosting can do the following:
1. If it is a dos, I would say you can ask your hosting provider to block it at server level.
2. If it is a ddos, you can use .htaccess authentication page so that only people can access your site until the ddos dies down. It is to help lessen the load on the server as static pages consume less resources compared to dynamic and bots tend to just hit your site so they won't get past the authentication page. But if it is a ddos, I wouldn't be surprised if the hosting provider asked you to go elsewhere if the ddos was too big and complex for them to handle. I moved a lot because of this.
3. If the ddos is too big and complex, consider going with a hosting provider that specializes in ddos protection. Servers at Awknet, Gigenet, and Staminus datacenters are a few big named ones. Usually they don't sell hosting, only dedicated servers but you can find resellers that host there.

People who have VPS's or dedicated servers can do the following:
1. Same as number 1 and 2 for people who have shared hosting which for number 1 you can do yourself.
2. Install ddos-deflate and make sure you have an firewall such as csf which provides some protection. I know there are other software solutions but can't remember them at this moment since it was a long time ago.
3. Consider using litespeed web server instead of apache. Works wonders for ddos's. Able to handle ddos's quite well than apache, especially the load and uses less memory as well. But it isn't free.
3. Consider hiring a ddos experienced admin to help out with looking through your logs to tweak your server to block dos and small level ddos.
4. Null route the ip if it gets big enough and just switch to a different ip and ride it through. Usually doesn't work now a days since ddos are trained on the domain than ip.
5. Consider moving to a ddos specialized provider as mentioned before.
6. Consider hiring specialized ddos protection service. Usually not in anyone's price range unless your a company. $$

Even with all these solutions if a ddos is large and complex enough, nothing can stop it, you just have to ride it out. Most you can do is move to a ddos specialized provider and have them try to help you block it which in most cases works to a good degree. Other people who get ddos also upgrade their hardware. Pretty much if you have enough resources, it doesn't really matter. These are just denial of service attacks, there are also bandwidth vampire attacks. Pretty much a slow attack where bots continue to hit a certain image or file causing you to waste bandwidth. You won't notice usually until you see your bandwidth consumption skyrocket. Leech Protect in Cpanel or .htaccess works well in these cases.

After my experiences with ddos's for a year, I just want to say that don't give them attention as in posting about them or anything like that. Just gives them reason to keep doing it. If you know who it is and you or they have issues, try to resolve any issues with them.

[*Salvo*]

Totally agree (unfortunately) +1
About litespeed, it isn't easy to run it, if the entire server is apache based. One has to re-study all, mostly if you've choosed (like me) to get an unmanaged server and do all by yourself, after crashes and tests.

Even with all these solutions if a ddos is large and complex enough, nothing can stop it, you just have to ride it out. Most you can do is move to a ddos specialized provider and have them try to help you block it which in most cases works to a good degree. Other people who get ddos also upgrade their hardware. Pretty much if you have enough resources, it doesn't really matter. These are just denial of service attacks, there are also bandwidth vampire attacks. Pretty much a slow attack where bots continue to hit a certain image or file causing you to waste bandwidth. You won't notice usually until you see your bandwidth consumption skyrocket. Leech Protect in Cpanel or .htaccess works well in these cases.

Sadly true.

[Lindy]

Well, coming from a person who dealt with a couple ddos, depending on how large and how complex the attack is, will depend on how you deal with them. It would also depend on what kind of hosting you have. People who have Shared hosting can do very little compared to say someone on a dedicated server due to the person having more control and able to implement protection against it.

People who have shared hosting can do the following:

1. If it is a dos, I would say you can ask your hosting provider to block it at server level.

2. If it is a ddos, you can use .htaccess authentication page so that only people can access your site until the ddos dies down. It is to help lessen the load on the server as static pages consume less resources compared to dynamic and bots tend to just hit your site so they won't get past the authentication page. But if it is a ddos, I wouldn't be surprised if the hosting provider asked you to go elsewhere if the ddos was too big and complex for them to handle. I moved a lot because of this.

3. If the ddos is too big and complex, consider going with a hosting provider that specializes in ddos protection. Servers at Awknet, Gigenet, and Staminus datacenters are a few big named ones. Usually they don't sell hosting, only dedicated servers but you can find resellers that host there.

People who have VPS's or dedicated servers can do the following:

1. Same as number 1 and 2 for people who have shared hosting which for number 1 you can do yourself.

2. Install ddos-deflate and make sure you have an firewall such as csf which provides some protection. I know there are other software solutions but can't remember them at this moment since it was a long time ago.

3. Consider using litespeed web server instead of apache. Works wonders for ddos's. Able to handle ddos's quite well than apache, especially the load and uses less memory as well. But it isn't free.

3. Consider hiring a ddos experienced admin to help out with looking through your logs to tweak your server to block dos and small level ddos.

4. Null route the ip if it gets big enough and just switch to a different ip and ride it through. Usually doesn't work now a days since ddos are trained on the domain than ip.

5. Consider moving to a ddos specialized provider as mentioned before.

6. Consider hiring specialized ddos protection service. Usually not in anyone's price range unless your a company. $$

Even with all these solutions if a ddos is large and complex enough, nothing can stop it, you just have to ride it out. Most you can do is move to a ddos specialized provider and have them try to help you block it which in most cases works to a good degree. Other people who get ddos also upgrade their hardware. Pretty much if you have enough resources, it doesn't really matter. These are just denial of service attacks, there are also bandwidth vampire attacks. Pretty much a slow attack where bots continue to hit a certain image or file causing you to waste bandwidth. You won't notice usually until you see your bandwidth consumption skyrocket. Leech Protect in Cpanel or .htaccess works well in these cases.

After my experiences with ddos's for a year, I just want to say that don't give them attention as in posting about them or anything like that. Just gives them reason to keep doing it. If you know who it is and you or they have issues, try to resolve any issues with them.

Great post.

Proper DDoS mitigation equipment that can handle higher line-speeds is prohibitively expensive - that's why there's companies that specialize in it. Most hosts do the same as we do: If it's a single dos, we will simply block it. If it's ddos, we will attempt to filter it. If it becomes too much to handle (that's a rarity, to be honest) we have no choice but to "null" the target customer/website to protect the integrity of our network for other customers.

[*Salvo*]

That was my server provider told me, Lindy: shut down the server and restart it tomorrow morning. I had a load (usually 0.01 - 0.05 - 0.03) that (what Nagios recorded) arrived til 680,00-700,00 -and a billion of queries made i don't know how. It is technically possible, unfortunately. it's a rarity, yes. There's not a tech way to discourage, other than .immortal hilighted properly.

[BlakeC]

I just want to say that don't give them attention as in posting about them or anything like that. Just gives them reason to keep doing it.

To me this is one of the the biggests thing you can do. Typically whoever is attacking will get bored after a few hours. Once they think they are getting to you though you can be sure they will keep it up.

[.Immortal]

Totally agree (unfortunately) [color="#556B2F"]+1[/color]

About litespeed, it isn't easy to run it, if the entire server is apache based. One has to re-study all, mostly if you've choosed (like me) to get an unmanaged server and do all by yourself, after crashes and tests.

[color="#1C2837"][size="2"] Sadly true.[/size][/color]

It is really worth it in my opinion if you are getting ddosed. Case in point, the WHT post litespeed highlighted http://www.webhostin...ad.php?t=616249 it was made by my admin who was helping me with my ddos attacks. He mentioned me in passing as the client who had the bandwidth attack. But I can vouch for its performance. It scales very well. PHP performance is pretty good but I like to use nginx to serve up static pages as it uses less resources. Litespeed is suppose to be a drop in replacement for apache, most things are compatible with it, but I found some things like modsecurity might be a little bit twitchy with it. Maybe that has changed from back in 2007.

Great post.

Proper DDoS mitigation equipment that can handle higher line-speeds is prohibitively expensive - that's why there's companies that specialize in it. Most hosts do the same as we do: If it's a single dos, we will simply block it. If it's ddos, we will attempt to filter it. If it becomes too much to handle (that's a rarity, to be honest) we have no choice but to "null" the target customer/website to protect the integrity of our network for other customers.

Pretty much SOP for any webhosting provider. In my adventure with dealing with my ddos's I went through my whole list in order until I couldn't go any further (renting mitigation appliances) I've had servers at all three ddos specialized datacenters and specialists such as them, they had a bit of a problem with my ddos since they were pretty complex.

I remembered another tip for us forum users. Make sure for guests you either have the search feature disabled or allowed to search every so often. I don't know how sphinx would handle it but if you just have traditional mysql searching and have search enabled for guests, they will use bots to spam your search until mysql or apache crashes. This was the start of my issues with ddos.

[Connor T]

I've been quite a target for DDoS and bandwidth rape attacks for quite awhile. As for the DDoS a combination of varnish, litespeed and tweaking IPB to not give cookies to guests lets a large influx of connections without bothering anything. As for the large scale attacks, I just ride it out.

To stop the bandwidth rape attacks I've used simple htacess rules to prevent direct viewing of images for those without a user-agent.

This was the month that I first experienced a bandwidth attack, and I was pretty bad at defending against it. Note on the 14th.





All the way to 18gb.

They hit my logo for I think around 140gb.

1 5009112 60.64% 150740277 77.37% /forum/public/style_images/clean/logos/p1.png

[TechModders]

se7ensins is under attack all the time by people that are paid by TTG... thats what they use:

http://www.serverorigin.com/

[*Salvo*]

This topic is now on my bookmarks, and a special thanks to .Immortal.
I want to learn more about litespeed (just in case..), cause if it may a bit prevent or limit the damages, I'll take immediately another temporary server and change totally mi actual setting(s).
I use this tool to hold under control the server status.

[7SiN]

se7ensins is under attack all the time by people that are paid by TTG... thats what they use:

[url="http://www.serverorigin.com/"]http://www.serverorigin.com/[/url]

I'm bumping this for a number of reasons. One being, my site was mentioned. I DID use ServerOrigin. I quickly got off of that. Their support was terrible, their monitoring was even worse, my site load times on average for Google Bot was 10-15s. These days, speed is being heavily weighed more and more by Google. By using this service, my SERPs dropped immensely. I went from 50+k uniques a day to 27-33k uniques a day. Many of my users were being blocked by their system so I would get multiple emails a day saying that they were IP banned. There were numerous problems I endured with them. I highly recommend users to STAY AWAY. Not only did the cause a lot of problems, they COST A LOT (solution below, keep reading).

After them, I tried SharkTech...aka "ddos specialists". My *beep*! lol Their monitoring system, from the sounds of it, is all their sites rolled into one. They weren't giving me the attack size of just my site but multiple of their sites so I was given number in the ranges of 30-45gbps attacks. I've been with AdminGeekz in the past and Scott told me we rarely ever got 100mbps and that big attacks RARELY go over 1gbps with any sites. So, not saying SharkTech are liars, I just don't have any logs of what my attacks were so my IP was constantly getting null routed. I moved away from them after we had a harddrive failure and while they were suppose to replace with a new one and set the "failed" one up as slave, they wiped it.

After Shark, I moved to BlackLotus. I used their dedicated servers thinking this would be best for the site. Well, similar experiences to ServerOrigin (maybe because they are now "buddy buddy" - I didn't realize they combined or something). On shark, my site loaded at 5s for Google bot, with BL, it loaded over 15s. Load times were horrendous, sometimes would take me 1-3 minutes just to load a page. When I needed to do stuff in the ACP, I would get a lot of time out errors. They also blocked a lot of our users. I had problems with Paypal connected and my IPNs coming through to upgrade my users automatically. Ugh...next to the worst experience so far. At least their support was nice (much nicer than SO), they were pretty speedy with responses, just not very good service for the price. OH, and they would also try to get you to upgrade to higher packages. I wanted to get reports of the attacks on my site, was going to cost me $150/month more. I complained that my users were getting blocked, they said that I needed some service that was $300/month more. I thought DDoS protection was to block DDoS, not users. I guess I was wrong.

NOW, I'm using DDoSDefend. They are coupled with AwkNet (only other proxy service I didn't use since their sales team is lazy and doesn't contact users). I have to tell you, I've been with them for about 2-3 weeks now (definitely enough time for me to give a good review after being with all those others for less than a month and knowing they were terrible), these guys are the BEST out there. Their prices are WAY below any of the other proxy services. I was one of the first to try this out since it's a brand new service and they knew of my hardship with DDoS (posted on multiple sites about the failures of these other companies). I asked him why his service was SOOO much cheaper and he told me that the others mark their services up so much that they become 7-8 figure companies over night.

We've not gone down ONCE for DDoS in the last 2-3 weeks. The load times are faster than EVER and my members love it. They have commented multiple times on how fast the site is and I get messages on my profile and over IM about how much they are grateful that I didn't stop until I found a solution that worked great! Since Google takes a few weeks to report what the load times are, I can't comment on that part but I can tell you, I've never been able to load my site as fast as this on any network we've been on. I highly recommend these guys.

They know their DDoS and I asked them about all these other solutions that people have brought up in this thread (nginx, litespeed, firewall, etc) and he had a great response to why those wouldn't work for people due to the types of attacks they get. So, if one of those worked for you, you were receiving the attacks that those could mitigate/block. For the people like me, you need a professional service. I'm not talking about a "DDoS specialist admin" like someone mentioned. This service is cheap and WELL worth the money. Since using them, my SERPs have climbed and I'm not 10+k uniques/day more than I have been doing the last couple of months.

Sorry for the wall of text...but for those that have suffered for 1-2 years from DDoS like I have, it's a Godsend when you find something that actually WORKS! Then you want to help other admins that have the same problem. I hope this reaches someone that can utilize it. Check their prices out, talk to support, ask them questions, I guarantee you'll be happy with this service.

(please don't lock this bc it was a "bump" - people need to be able to read and comment on this)

[Gary.]

Nice post, And well explained !

What I suggest to anyone is if your machine is running fine, Good uptime, And copes very well when not under attack then do not move host!

Your best bet is to ask your webhost for permission to contact myour DC via email and enquire about putting a hardware firewall infront of your box and have it configured to filter such attacks, Its very very easy to do on CISCO boxes, The menu is "idiot proof" and you cannot go wrong. Now depending how you speak to them you could avoid a setup fee, Or if not you can ask them to charge you monthly but, You could pick up a Cisco ASA 5510 firewall directly from the DC at a cost of around 60USD per month.

Now no software will ever stop DdoS attacks ! Only DoS can be filtered via software / script. If you cannot afford the cost of a firewall then the best thing to do is ask your host, To contact the DC and nullroute the IP's connecting to your machine, Now this will stop all the attacks, But will make it sluggish for users.

[7SiN]

Nice post, And well explained !

What I suggest to anyone is if your machine is running fine, Good uptime, And copes very well when not under attack then do not move host!

Your best bet is to ask your webhost for permission to contact myour DC via email and enquire about putting a hardware firewall infront of your box and have it configured to filter such attacks, Its very very easy to do on CISCO boxes, The menu is "idiot proof" and you cannot go wrong. Now depending how you speak to them you could avoid a setup fee, Or if not you can ask them to charge you monthly but, You could pick up a Cisco ASA 5510 firewall directly from the DC at a cost of around 60USD per month.

Now no software will ever stop DdoS attacks ! Only DoS can be filtered via software / script. If you cannot afford the cost of a firewall then the best thing to do is ask your host, To contact the DC and nullroute the IP's connecting to your machine, Now this will stop all the attacks, But will make it sluggish for users.

Most of the time the DC will nullroute your IP for 24 hours (depending on the host too) until the attack stops. Not sure how Cisco firewalls perform and how large of attacks they can filter...or even if it affects the performance of the server (load times for end users). But DDoSDefend has a great set up + increase performance of your server, plus it's only $65/month for the lowest package.

I would suggest everyone to try what works best for them. As you can see by my previous post, I've been around the block a few times and this is the only thing that has worked for me.

[Lindy]

Why are you getting DDoS'd so much? Gaming website?

[7SiN]

Yeah, gaming site. Kids seem to think they are cool when they can put a site down. All large gaming sites that I've talked to get massive DDoS attacks. Not many of them have found a solution yet and I'm working on getting them on this service to see if we all come up with the same conclusion (that it's AMAZING).

[Lindy]

Gamers do attract much drama, unfortunately! :) I personally have a zero tolerance policy at IPS in terms of drama. If I didn't... we could literally spend all day dealing with "xyz gamer is threatning to DDoS me. here's proof!" Although we do have our own equipment and datacenter space, we do not consider ourselves a web host - we're a solution provider. As such, at the first sign of service-impacting drama, we null and respectfully ask the target to make other arrangements elsewhere.

I'll investigate that service and if it's worthwhile, I have no issue considering it as a recommendation to our clients. I don't like to push anyone away without an alternative.

[7SiN]

Well, I also don't allow drama...which sometimes creates raging lunatics. Banned members think they did no wrong (when we have a specific system of banning and don't just ban you for no reason and usually not right away, we utilize the warning system), they are from a "competitor" site, they are from a copy & paste site, or they are members that think they deserve a staff position and go and create their own site and "try to take us down".

We don't even get threats anymore, they just do it and we have no clue who is doing it. That's the bad thing with running a board with almost 300k members. :/

[Gary.]

The cisco AS basic will filter around 0.5GB with 50,000 concurrent connections, It will basicly block 98% as by time the next batch comes along the traffic heading towards you has already been split and blocked.

Now most company's who do offer DdoS protection hosting, Are only doing in what you do not currently understand or have knowlage to do, Thats stick a hardware firewall in front of your server and setup filters.

Have a small read at the hardware I mentioned above.

http://www.cisco.com/en/US/prod/collateral/vpndevc/ps5729/ps5713/ps4077/data_sheet_c78-459520_ps6120_Products_Data_Sheet.html

[Robulosity2]

This is why I like our F5 Devices :D

On a serious note, honestly if you have the budget for hosting a 300K member site (if it's actually highly active to require a decent dedicated machine) it's a good idea to look into rented/leased Hardware firewalls even if its through the rack space provider

[7SiN]

This is why I like our F5 Devices :D

On a serious note, honestly if you have the budget for hosting a 300K member site (if it's actually highly active to require a decent dedicated machine) it's a good idea to look into rented/leased Hardware firewalls even if its through the rack space provider

I've done my research and used hosting companies that have had such firewalls. It doesn't work.

And yes, my site is active. For roughly 12 hours out of the day, we have over 1,000 active users online within a 5 minute period (setting it to 15 would make the screen stretch and I hated scrolling through all that lol).

Solutions need to be found that can filter nearly ALL types of attacks (since most people think there is only 1 type of "DDoS" out there) and it needs allow legit traffic through without blocking any of them. Again, the only thing I've found to do all of this as well as increase the performance of my site has been DDoS Defend.

I wish the owner could come in here and comment.

[Gary.]

There is DoS, Which is tons of connections from a single IP, Then there is DdoS which is connections from multiple IP's.

It can be easily filtered, First of you need software based to filter out the IP's what are connecting with 20 per second, Anything less would not affect the server if keepalive was on to watch real traffic.

Then you use the hardware firewall to filter more than 20 per second, The main type of attacks, Mostly botnets are HTTP:/01 where it just hammeres the hell out of the IP, Or you get the TCP / UDP what in theory looks like port bashing as when you watch them come in the ports are all over the place, Again, A simple block.

Combination of hardware and software, You can be sure you would kill 98% of the bad traffic, Only way the Legitimate users would be banned as if there AOL, where there proxys could be in the range of what the attack is.

Hense, I don't see how AOL is still going with all proxy server being null routed, I'm supprised there connections work in EURO ! and outside, But you can easily see what legit users are getting blocked via the logs so you find another work around.

[.Nuno.]

Hello,

I was reading your posts and remember a blog posts I've read last year that could help mitigate the issue ... a frontend farm with nginx + upstream hash + varnish hiding the backend('s) could alleviate your problem.
With a TTL of 10 minutes , your backend only have to deal with 1 guests request each 10 minuts for each page.
This setup wont stop a DDoS but will help you deal with thousand of hit's per second ...

This is an excerpt from the wordpress post:

"We are currently using Nginx 0.6.29 with the upstream hash module which gives us the static hashing we need to proxy to varnish. We are regularly serving about 8-9k requests/second and about 1.2Gbit/sec through a few Nginx instances and have plenty of room to grow!"

[Robulosity2]

I've done my research and used hosting companies that have had such firewalls. It doesn't work.

And yes, my site is active. For roughly 12 hours out of the day, we have over 1,000 active users online within a 5 minute period (setting it to 15 would make the screen stretch and I hated scrolling through all that lol).

Solutions need to be found that can filter nearly ALL types of attacks (since most people think there is only 1 type of "DDoS" out there) and it needs allow legit traffic through without blocking any of them. Again, the only thing I've found to do all of this as well as increase the performance of my site has been DDoS Defend.

I wish the owner could come in here and comment.

They work fine, if you have a scalable system and set it up properly... There are between 10-40 DDOS attacks on our network per day, not one has succeeded in 5 years