Hello folks. Got an devops related issue for an EE site that I need to know if I'm on the right trac

Have you got any search engine crawlers or other bots hammering the site?

Are you saying you allow (up to) 4mb images on the front end?

The images being uploaded are up to 4MB, I don’t believe they stay that big when JCOGs transforms them. Looking at one now that they uploaded that was 2MB, it’s under 100kB after JCOGs did its magic. THey’re keeping them mostly under 1MB after I advised them. Mostly.

We did recently just block the Amazon bot, it seemed to appear the most in the logs out of all the others. And also blocked one for WeChat.

recently blocked as in those were what I added after yesterday’s outage.

There’s also a couple of /24 IP blocks that belong to MSFT that I outright blocked too, before this, as they were hammering the heck out of the site (100k requests in 24 hrs??)

I’m wondering if you block all crawlers/bots for an hour or so what would happen then.

Oh we did that yesterday with the “we’re under attack” mode on Cloudflare; as far as I know, that blocks all bots & challenges humans to prove they’re not a bot.

Then I blocked the two bots I mentioned, and opened it back up after like six hours. It’s been fine since but my gut feeling is this is just another lull before it happens again. I’ve been chasing this for about a month so far. We even bumped their droplet up to double the RAM just in case that would help (it didn’t).

DB on the same box? Forge doesn’t do a super job on configuring the DB (and doesn’t update the config after you update the server with resources)… ALSO. any shot you can list other add-ons in use?

It is deployment through Ploi, but yeah the dev & main db are on the same box.

BUT! Upgrading everything helped immensely (especially getting both dev/live updated in both respects). I suspect it was a combination of PHP7.4 and EE7.4x, because while the CPU seemed to settle slightly after the PHP bump, it wasn’t until I got EE on the latest version to where the CPU is finally very happy. On TOP when it was PHP7.4, the spikes between PHP/MySQL were frequent. Several times in a ten minute span.

Now though, MySQL’s process isn’t surpassing 30% in normal visits. The client is going to be releasing a newsletter on Wednesday, I’ll be watching to see how it handles the traffic surge as that’s another thing that had caused the site to crumple in the past.

But here’s a screenshot from the DO graph in the hour before, during and after the PHP/EE upgrades (specifically for the production site - dev was already done). The spiking in TOP was never long enough to show up on their “averaging” on the graph, but it kind of shows in the peaks on the before.

/asset/images/dovetail/14e6d4ac819183819635f502c7b43b48.png

In the past 24hrs there’s been one spike up to 55% on their graph, but it spends more time under 25% than over it now. That’s quite a lot better than it was before the upgrade.

And here’s to show it over the last week. The one that tops out between the 5th & the 6th was the last “outage”.

Quite the visual for proving upgrading can only be put off for so long. 😉

/asset/images/dovetail/6f32d14b7e6fde34ce7b15c3ad1c8602.png

ah, interesting…

Theres actually a good number of performance improvements in the last few releases as well.

Update post newsletter release: The site handled the traffic surge - about 4k visits immediately following the release - perfectly. Beforehand the site would’ve likely gone unreachable again. So, upgrading both PHP & EE in this case appears to have been a solid solution. I’ll of course continue to monitor things as there can always be a rush of bots that could cause issues in the future, but I feel better that I’ve done quite a bit to help mitigate issues. These upgrades also helped the site speed in general, so that’s an added bonus.