EE redirects used by bots!
Development and Programming
Hi,
One of our many EE websites had trouble with a bot bugging them: It was automatically redirecting urls many, many times using {website_domain}?URL={their_spam_url} . Apparently this is standard EE functionality which can be traced back to system/ee/legacy/libraries/Redirect.php . The only options we can set using the config setting force_redirect (boolean), causing the redirect to go automatically or by a buttonclick. Is there any logical explanation why it works this way? Am I doing something wrong?
Kind regards, Zef Oudendorp
Hi Zef,
Did you set the rank denial setting (bottom) on this page: /cp/settings/security-privacy
We’ve talked about this a few times internally. We never want to continue doing something just because it’s always been that way, but we also are hesitant to remove functionality that could be in use by many many sites out there. In 6.1.0 and 5.4.3 we released an update on this to where the redirect warning page now has no index, no follow . This means that while bots can still redirect through your site, they get no SEO value from doing so. Before those updates, spammers would use this feature to have redirects to their sites appear in Google and other search engine results when users searched for keywords related to your site.
I tested this on my installation after reading this since I was worried it could be used for phishing or malware, but EE does show a warning message and the user has to click the Continue button in order to proceed. At least in my installation, the redirect is not automatically.
Speaking of that message. Can this be customized? I assume it’s a system template. I was not even aware this was an EE feature. It could be useful to show a warning when users are leaving the website for compliance reasons, privacy or security while leaving authenticated sections.
Is there way to disable/ turn off the ‘?URL= redirect’ functionality or remove the “continue” button with the link on the Redirect Warning page? Instead of showing the Redirect Warning, we would rather just show them our main page or a 404 error page.
We have been getting a lot of link spam from bots to XXX pages over the last few weeks using the URL redirect functionality, and trying hard to get this to stop.
I.e. So if someone typing in https://www.agricover.com/?URL=https://expressionengine.com/ we don’t want them to get to the redirect page in the attached screenshot, but just want to show them: https://www.agricover.com/
Kind regards, Knut Ellingsen
@Agricover. For now several users have reported success using the following in their .htaccess file (if you’re on Apache)
RewriteCond %{QUERY_STRING} ^URL\=(.+?)(-[0-9]+)?$
RewriteRule ^(.*)$ /index.php? [R=301,L]that would redirect the redirect page to you homepage.
In 6.3.0, we’re planning to release the ability to turn this functionality off completely. You can review the Pull Request here and even pull it down and try it out if you’d like: https://github.com/ExpressionEngine/ExpressionEngine/pull/1857
Hi Andy, I tried out your .htaccess file solution. This worked great. Thank you so much for quick response.