It works, but sometimes a little too well.
There are good bots and bad bots. The bad ones wreak havoc on your website.
Cloudways, a platform for managing sites hosted on cloud platforms, recently added Malcare’s bot protection to its service. It promises to block malicious traffic, protect your website from attacks, and reduce server resource usage.
I turned it on to see if it delivered. In short: yes, it blocks a lot of bad stuff, but it also throws the baby out with the bathwater if left unattended.
For example, it blocked the Stitcher podcast service from accessing my podcast feed. It also blocked a lot of public relations services, such as Meltwater, from accessing the site. Services like Meltwater ensure that what I write is read by more people.
On the other hand, it helpfully blocked Ahref’s deluge of bot attempts. I can’t see any benefit of Ahrefs to site owners; it only seems to help other people with their SEO.
Bot protection allows you to whitelist IP addresses and bots that it decides to block, but I’d prefer to work on a blacklist basis instead. In other words, show me a list of bots and I’ll select which ones to block. It would also be nice to block access by known IP addresses to specific resources, such as XMLRPC, which is frequently abused.
I also found that Cloudways had trouble loading detailed data about bot attempts. It took several days of checking the dashboard to get it to actually load the data. Even then, it worked only part of the time.
One helpful feature of bot protection is that it also blocks WordPress login attempts. According to the dashboard, it’s blocking about 2,000 invalid login attempts every day.
The net-net is that bot protection on Cloudways is useful but a blunt tool. You need to work with it to make sure it doesn’t block helpful bots from reaching your website.
Steve says
“2,000 invalid login attempts every day”
Andrew — this seems like a huge number. Do you know if this level of attempts is common on operating websites?
Lifesavings.online says
You need to change your login URL from /wp-admin and admin.php
There are plugins for that.
If people don’t register to your site, there’s no sense having a user-login at all. Ur just letting bots waste server resources.
Bot *WILL* look for a login page. /login /admin etc.
Next, I personally ban any user that hits 3x 404’s within 1 minute…This segways into what you wrote the other day…IF you don’t want to ban GOOD crawlers – you can’t have any hard 404 on your website!
You can start banning more bots. Google sees an understands what u are doing – this all helps SEO
I don’t believe any legitimate user with good attention are going to run into 3x 404 on you site. Those that do are doing things like looking for that login page.
Andrew Allemann says
It’s a good idea. I’ve done that one what site and should do it on this one, too.