Filtering Unwanted Nuisance Web Requests

Support for security such as Firewalls and securing linux
Post Reply
Al_Stu
Posts: 52
Joined: 2010/09/14 21:05:16

Filtering Unwanted Nuisance Web Requests

Post by Al_Stu » 2020/06/03 00:24:22

Website gets a lot of request for "/webalizer/...", "/awstats/...", and "/wp-...".
Even though they do not exist, 404 not found, the requests keep coming for years.
The requests come from different addresses that are infrequently reused. There will be a batch of requests (dozens in a few seconds) from an address and then that address will not be used again. At least not for a long time.
Nearly all addresses are from RIPE.

Currently using these iptables rules to filter them out. Ceasing all further communication with the offending client until quiescent for 24 hours.

Code: Select all

-A PREROUTING -m recent --rsource --update --seconds 86400 --name DYN_DROP_LIST -j DROP
-A OUTPUT     -m recent --rdest   --update --seconds 86400 --name DYN_DROP_LIST -j DROP

-A HTTP_DROP -m string --string "GET "    --algo bm --from 40 --to 90 -j STR_CHK
-A HTTP_DROP -m string --string "POST "   --algo bm --from 40 --to 90 -j STR_CHK

-A STR_CHK -m string --string "webalizer" --algo bm --from 40 --to 200 -j DYN_DROP
-A STR_CHK -m string --string "awstats"   --algo bm --from 40 --to 200 -j DYN_DROP
-A STR_CHK -m string --string "wp-"       --algo bm --from 40 --to 200 -j DYN_DROP

-A DYN_DROP -m recent --rsource --set --name DYN_DROP_LIST -j DROP
Are there better ways?

Obviously this method will not work with SSL/TLS.
Fortunately this site does not currently support SSL/TLS. But in the future perhaps it will.
Will iptables ever support decryption, similar to what Wireshark does, by providing the private key?
Last edited by Al_Stu on 2020/06/04 21:36:52, edited 4 times in total.

User avatar
TrevorH
Forum Moderator
Posts: 29648
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Filtering Unwanted Nuisance Web Requests

Post by TrevorH » 2020/06/03 00:52:44

It's just skript kiddies running automated tests for known vulnerabilities in old copies of those packages. If you don't have them installed they'll 404 and move on. Your best bet is probably to look at fail2ban and use that to watch your logs and ban the ip addresses that way.
CentOS 6 will die in November 2020 - migrate sooner rather than later!
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 is dead, do not use it.
Full time Geek, part time moderator. Use the FAQ Luke

Al_Stu
Posts: 52
Joined: 2010/09/14 21:05:16

Re: Filtering Unwanted Nuisance Web Requests

Post by Al_Stu » 2020/06/03 01:31:46

I know they are script kiddies. But they are a log spamming nuisance. Suppose I could write a script to purge those log entries. Prefer to keep them out of the system to being with. The iptables rules seems to be effective but don't know if maybe better ways. It certainly won't work with SSL/TLS.

They don't move on from a 404. They just incessantly continue for years.

Have explored the banning method. Banning them is of little use due to the addresses being infrequently reused. Also by the time an address gets picked up from the log files it's no longer being used. They are done with that address (at least for a long time). That block list becomes enormous and due to infrequent address reuse infective. End result would probably be pretty close to banning all of RIPE. :( Seems like RIPE is a giant botnet.

Post Reply

Return to “CentOS 7 - Security Support”