On Tue, Aug 2, 2011 at 4:49 PM, Ron Guerin ron@vnetworx.net wrote:
Sam Rudge wrote:
This URL has never existed, not 'did exist but has now been deleted' because we don't fully delete things from our database just mark them deleted, this URL has never forwarded to anything other than our 404 page.
Another example of the other behaviour is this
Please remove the abused shortner: http://dft dot ba /-NqD
[etc]
This URL did exist (but has now manually been deleted), but forwards to the domain 'li.ru', not blacklisted by SURBL. Trying to access the URL by any methods from our server (CURL, WGET etc.) returns a 500 server error so it looks like the site has blocked us from automatically figuring out where the URLs are redirecting to (I guess on an IP based block, it works from other servers). If SURBL isn't going to blacklist sites why are we being alerted that the link is being abused.
Our web host says SURBL often generates "false positives that should be ignored" but I'm trying to avoid our site getting blacklisted/flagged etc.
Any suggestions?
I also run a URL shortener (Also known as "A dozen lines of PHP to shorten, and thousands of lines of anti-abuse code"), have had some run-ins with blacklists including this one. So far, I have found SURBL is the only one of the bunch that wasn't tedious or outright impossible to deal with or obviously running with no one minding the store. (My favorite was someone I know who works for a URL shortener, telling me they found themselves blacklisted and were given links to their own best practices documents (which they were following)) as a resource to avoid being listed.
What you're probably soon to be told by someone is that you really didn't want this list, and that you wanted to send an email message to whitelist (at) surbl dot org. That said, you'll definitely want to get to the bottom of the non-existent links thing. It's not out of the realm of possibility that these links which were never valid were nevertheless, used in some kind kind of spam. It's also within the realm of possibility that someone's trying to get you listed. The former much more likely than the latter, but you'll want to find out.
http://en.wikipedia.org/wiki/Joe_job
- Ron
Perhaps the abusers have code that creates a shortened link but doesn't check that it works, and they spam the shortened link anyway whether it works or not.