Comments on Block Referrer Spam (Updated) - ILoveJackDaniels.com

Comment on Block Referrer Spam (Updated)

Comment by Bill ( http://freebsdrocks.net )

I geta TON of spams on my FAQ pages under comments. I have tried to implement this on my site but I can't seeing as how I use vhosts. Any way around this?

Comment on Block Referrer Spam (Updated)

Comment by Vasilii ( Vasilii )

Sentimental and nostalgic. Great.

Comment on Block Referrer Spam (Updated)

Comment by Karma Debugger ( http://www.linklog.org )

I had a wrong link coming in from a wrong place. This technique was the cure.

Comment on Block Referrer Spam (Updated)

Comment by Rob ( http:// )

I though some folks might like to take a look at this site for referrer spam:
http://unknowngenius.com/blog/wordpress/ref-karma/

he wrote a neat php script to automate updating referrer blocks.

Comment on Block Referrer Spam (Updated)

Comment by Rob ( )

Hey folks, sorry for the spam here but i'm wondering how you can tell if this is working? I found a couple sites referring to different ways to do what you're saying. One is a .conf file in /path/to/conf.d/ where it works w/ apache and other is .htaccess. What i'm seeing is a ton of referrer spam in our access logs which doesn't belong. It's forged as it's asking for sites that our apache server doesn't host. I would like to get this out of my log file and put the kabosh on the spamming offender by simply blocking their access (if possible). Any suggestions here?

Comment on Block Referrer Spam (Updated)

Comment by CortalUX ( http://cortalux.co.uk )

Dammit, I really hate spam.

Comment on Block Referrer Spam (Updated)

Comment by Julius ( http://jult.net/txt/blocks )

But still, how are you gonna catch ref.spam with just a list of descriptions? It's an endless road I'm travelling, and I'm getting quite fed up with these idiots doing this.

# the biggest losers ever ( they can't even spell: )
RewriteCond %{HTTP_REFERER} (-nude\.org) [NC,OR]
RewriteCond %{HTTP_REFERER} (abrianna\.org) [NC,OR]
RewriteCond %{HTTP_REFERER} (amanti\.org) [NC,OR]
RewriteCond %{HTTP_REFERER} (anali\.org) [NC,OR]
RewriteCond %{HTTP_REFERER} (burdizzo\.org) [NC,OR]
RewriteCond %{HTTP_REFERER} (bucetinhas\.) [NC,OR]
RewriteCond %{HTTP_REFERER} (calcinha\.) [NC,OR]
RewriteCond %{HTTP_REFERER} (cogidas\.) [NC,OR]
RewriteCond %{HTTP_REFERER} (esibizioniste\.) [NC,OR]
RewriteCond %{HTTP_REFERER} (fimosis\.org) [NC,OR]
RewriteCond %{HTTP_REFERER} (folladas\.) [NC,OR]
RewriteCond %{HTTP_REFERER} (gotico\.) [NC,OR]
RewriteCond %{HTTP_REFERER} (gozadas\.) [NC,OR]
RewriteCond %{HTTP_REFERER} (hargitay\.org) [NC,OR]
RewriteCond %{HTTP_REFERER} (loredana\.org) [NC,OR]
RewriteCond %{HTTP_REFERER} (mamando\.org) [NC,OR]
RewriteCond %{HTTP_REFERER} (minifalda\.org) [NC,OR]
RewriteCond %{HTTP_REFERER} (plumprumps\.) [NC,OR]
RewriteCond %{HTTP_REFERER} (porono\.) [NC,OR]
RewriteCond %{HTTP_REFERER} (ramalan\.org) [NC,OR]
RewriteCond %{HTTP_REFERER} (stretched\.org) [NC,OR]
RewriteCond %{HTTP_REFERER} (subsonica\.org) [NC,OR]

Comment on Block Referrer Spam (Updated)

Comment by Bruce ( http://www.synysys.com )

I've been running a somewhat modified system to what I documented in comment #31. It's currently blocking 3208 IPs from people who have behaved badly on my system (mostly attempts at referrer spam, but that number also includes SSHD and misc script probes against my HTTPD) The firewall easily handles this many blocks and my web server is much happier with the reduction in load.

The system responds in real time to these attacks, gives them a custom 403 Error page and then blocks their IP. The custom 403 Error page is for non-script users who may be blocked inadvertently. It has a link to a recovery system which unblocks their IP and restores their access. Of course a bot doesn't follow the on-screen instructions and even if it did, it would just get blocked when it started behaving badly. All in all, it seems to be quite an effective solution.

For folks who don't have admin access to the firewall
on their server, the system is still quite effective, but you will continue to see all of the attempts in your httpd logs.

If anyone is interested in further details, I'd be more than happy to discuss this via chat or email. You can contact me on www.synysys.com. Anything that we can do to slow these idiots down is a step in the right direction.

Bruce

Comment on Block Referrer Spam (Updated)

Comment by Srinath ( http://www.srinath.info )

Thx ! was really useful !!! i had like 100 referrr spam daily. !!!

Comment on Block Referrer Spam (Updated)

Comment by Bruce MacKay ( )

My daughter maintained her robotics blog on my site (http://synysys.com/roboblog) for several months until the project came to conclusion and was eventually taken off line. During it's life span, it fell to a bit of neglect and became the target of the referrer spam bots. Today our site still gets over a thousand hits per day looking for the old exploit.

I know this is a bit like closing the barn door after the cows have all run off and perhaps in this case even part of the barn burned down, but I thought I'd share the solution I hacked together today. I wasn't satisfied with a purely mod_rewrite solution since as others have noted, you still get a one line log entry in you access log. Essentially my solution is a two pronged approach. First it uses mod_rewrite to redirect the spammer back to their own machine. Second it puts a DROP entry in my firewall so that they won't be coming back to visit again any time soon. That way my logs aren't filling up with the same old rewrites over and over.

The entry in httpd.conf looks like this

RewriteEngine on
RewriteCond %{QUERY_STRING} disp=stats
RewriteMap referer-deny prg:/etc/httpd/refererdeny.pl
RewriteRule ^(.*)$ ${referer-deny:%
{REMOTE_ADDR}}/BITE_ME_SPAMMER? [R,L]

In my case it was a particular query string that typified the bulk of the spam traffic, but you can add other patterns to the above rewrite conditions to suit your own needs.

The PERL script looks like this

#!/usr/bin/perl
$| = 1; # Turn off buffering
while () {
print "HTTP://",$_;
$b = ("/sbin/iptables -A INPUT -j DROP -p tcp --destination-port 80 -s $_");
system ($b);
open (OUTFILE,">>/etc/httpd/referer.deny");
print OUTFILE ("$b");
close (OUTFILE);
}

The referer.deny "log" looks like this

/sbin/iptables -A INPUT -j DROP -p tcp --destination-port 80 -s 219.93.21.20
/sbin/iptables -A INPUT -j DROP -p tcp --destination-port 80 -s 220.165.140.8
/sbin/iptables -A INPUT -j DROP -p tcp --destination-port 80 -s 83.100.149.29

So you could easily add #!/bin/sh to the head of it and run it as a shell script separately if you wanted. However, before you do that, you should probably sort the file and remove any duplicate entries that may have crept in. I have chosen to only block access to port 80. You could easily add port 25 or even remove the destination-port all together and completely block them from your site. Just be aware that some clever fool could forge your IP and potentially block you from your own site. Of course you could reduce the output to the log file by substituting $_ for $b and just end up with a list of blocked IPs.

I realize that not all site admins have root access to be able to run the firewall commands, so you might modify this to update a hosts.deny file that you've defined in your own .htaccess configuration. The point is you don't really want to have to manually enter every IP or host name if you are really getting bombarded. Again if you do this, you'll probably want to sort the file and remove dupes. I'd also recommend that us the DB utility to speed your lookups if you end up with a significant number of blocked hosts. You really don't want to bog down your site with lookups on account of these spammer fools.

Of course one of the problems that I alluded to earlier is that you may end up with unwanted blocks defined in your system. Most hosting environments offer CRON access. You might choose to flush the firewall rules over some period. Many of the spammers are running client based tools from dynamic IP pools on the ISP. Over time you could end up blocking a significant number of IPs that were only used once against your system. Since this system is automated, it's probably safer to clear it out periodically and let it repopulate itself with the bad apples that keep coming back.

I hope this helps someone. It seems to be working wonderfully for our site. My daughter's robotics project was archived as a PDF for those who are looking for it and the spammers trying to exploit the referrer logs aren't stealing my bandwidth or chewing up file space with senseless logs any longer.

Bruce