Nginx : Block Referrer Spam

In this article, we will show you how to block referrer spam in one of our Nginx web server.

1. Find the Patterns

Check the Nginx access.log file, and identify the “referrer spam” patterns.


$ sudo tail -f /var/log/nginx/access.log

Some patterns :


200 http://???.ru/engine/redirect.php?url=http://mywebsite.com/site/blogspot.com.au
200 http://???.com/tp/out.php?link=alternatevideo&url=http%3A//mywebsite.com/site/readyliftproshop.com
200 http://???.edu/online/redirect.asp?url=http://mywebsite/site/wheretoshophongkong.com

We are going to block following patterns :

  1. redirect.php
  2. out.php
  3. redirect.asp

2. Block & Return 405

Edit sites-enabled/default, if any of above patterns is matched, return 405 directly.


if ($http_referer ~* (redirect.php|redirect.asp|out.php) ) {
     return 405;
}

Review the full example in our Nginx web server, actually we block a lot of spam’s patterns.

/etc/nginx/sites-enabled/default

server {
  listen          80;
  server_name     mysite.com;
  root            /etc/tomcat7/webapps/mysite;

  proxy_cache one;

  if ($http_referer ~* (seo|referrer|redirect|link=|url=|url?|path=|dku=|babes|click|girl|jewelry|love|organic|poker|porn|sex|teen|video|webcam) ) {
      return 405;
  }

  location / {
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Server $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://127.0.0.1:8080/;
  }
}

Done, restart Nginx.

Check the log file again, now, any future requests that are matched the declared patterns, 405 will be returned.


$ sudo tail -f /var/log/nginx/access.log

405 http://???.ru/engine/redirect.php?url=http://mywebsite.com/site/blogspot.com.au
405 http://???.com/tp/out.php?link=alternatevideo&url=http%3A//mywebsite.com/site/readyliftproshop.com
405 http://???.edu/online/redirect.asp?url=http://mywebsite/site/wheretoshophongkong.com

References

  1. Nginx.org : Referrer Spam Blocking
  2. Blocking Referer Spam
  3. List of HTTP status codes

About the Author

author image
mkyong
Founder of Mkyong.com, love Java and open source stuff. Follow him on Twitter, or befriend him on Facebook or Google Plus. If you like my tutorials, consider make a donation to these charities.

Comments

avatar
2 Comment threads
0 Thread replies
0 Followers
 
Most reacted comment
Hottest comment thread
2 Comment authors
utuxiaanon Recent comment authors
newest oldest most voted
utuxia
Guest
utuxia

is it possible read the list of referrers to ban from a file?

anon
Guest
anon

Is it actually valuable to track and publish referrer info? Can we get some more information about referrer spam and the underlying use case?