On Sunday, May 8, 2005, 6:28:39 PM, Matthew Wilson wrote:
Does anyone know of a SA rule to check how recently a domain name has been registered?
The various uri lookups catch the vast majority of spammy urls during the day, but from 2-5 a.m. CST, my servers get hit with tons of spam with urls that aren't in SURBL yet. All of the domains are newly registered domains (registered in the past week or so).
I know that the SARE ninjas have some private tools to do this kind of lookup for their feeds and manual lookups, but I'm wondering if this kind of thing could be worked directly into a SA rule.
This idea had been talked about on the SA Users list, but the SA folks did not want to develop and maintain a database service of domain ages. Determining the age can be non-trivial, as is providing a data service. Therefore it's probably not something that would lend itself to an SA rule directly. Certainly we would not want each SA installation to be doing whois queries independently. That could overload the various whois servers.
However domain age is definitely a good indicator of spammyness. Generally speaking, the older a domain is the less likely spammers are using it. Many spam domains are are very recently registered, for example a few days ago.
Probably a better approach would be for us to look at some of the CBL spamtrap URI domains, check their ages and some other factors on the SURBL data side, and list them in the new SURBL XS list if they meet the appropriate criteria. This is in the works and on my list of things to do. Probably it will work very well at detecting fresh spams like some of the ones you've been spotting.
Age bias could also be applied to other lists such as SC. It's already part of the OB list in the fact that Outblaze won't put domains older than 90 days on OB. That catches a lot of spammers and tends to prevent a lot of FPs.
Jeff C. -- Don't harm innocent bystanders.