502 and 504 errors shortly after startup

[Aaron M]

We're hosted on AWS with load balancing. We started getting hit with DDoS earlier today and our site would go down for a couple minutes while it scaled up another instance. This stopped working at a point though and we're just getting hit with 502 and 504. We scaled down to one instance and rebooted, came up for a minute and the site worked, then it went back to 502's and 504's. 

The admin cp stays up for longer (although running slow) but the frontend starts blowing errors very quick.

We've tried clearing cache but to no avail...

Our team member who set all this AWS stuff up left us abruptly so we are flying blind and down on our busiest day of the month :*(

Really need support, happy to pay someone whatever hourly rate to hop on a discord call and help us out.

Please let me know any suggestions or thoughts. Thank you.

This is an error we're getting via nginx:

 

[Aaron M]

Another couple errors from the system logs:

[Charles]

502/504 errors are AWS for "could not get a response from your EC2 in the timeout timeframe."

Those "please reduce your rate" are S3 errors. It's hitting S3 very, very fast if you are seeing that.

You should look at WAF in front of your setup to mitigate DoS attacks. It's very good with this sort of thing as you can set rules for flood control.

[Aaron M]

Thank you Charles for the reply.

We're looking into WAF settings now.