-----Original Message----- From: Frank Ellermann [mailto:nobody@xyzzy.claranet.de] Sent: Thursday, September 09, 2004 10:01 PM To: discuss@lists.surbl.org Subject: Re: [SURBL-Discuss] Whitelist Please
Jeff Chan wrote:
Chris and Ryan and Raymond, don't even think about proposing a subdomain list. LOL! ;-)
What's the problem with this idea ? It would be only one level above the real host, so for say claranet.de you would have to consider www.claranet.de and xyzzy.claranet.de, but you would ignore www.xyzzy.claranet.de or more.levels.xyzzy.claranet.de
Then if I start to spamvertize my site you catch me without hitting any other user.claranet.de (let alone www.claranet.de)
Assuming that my ISP doesn't neeed weeks to cancel my account after I started to spam the xyzzy entry will expire soon.
It's about time ICANN cracked down on rogue registrars.
I'll believe it when I see it. These registrars pay ICANN's budget, don't they ?
There will always be disagreement about that optimization point. That is natural. (It's also a PITA.)
Sometimes your criteria appear to be a bit obscure for me. Of course some people may love a "joke of the day" mail - that's okay, if they like it they won't report it as spam.
But others don't like any unsolicited jokes, and they would report it as spam. In that case the joke-of-the-day site _is_ spamming, and it's okay to list them. Even if they also have some real fans with a "legit" interest in their joke of the day. In that case you can't avoid a collateral damage, whatever you do. Bye, Frank
hmmm.... can't we treat these like we treat com.ar tld? or co.uk? Like Frank said, just checking the subdomain one more level up for these guys. I don't see the harm in that. Or am I missing something again?
I think what Jeff meant was another SURBL list entirely ;) No, I think we have enough as well.
--Chris
On Friday, September 10, 2004, 7:43:51 AM, Chris Santerre wrote:
-----Original Message----- From: Frank Ellermann [mailto:nobody@xyzzy.claranet.de] Sent: Thursday, September 09, 2004 10:01 PM To: discuss@lists.surbl.org Subject: Re: [SURBL-Discuss] Whitelist Please
Jeff Chan wrote:
Chris and Ryan and Raymond, don't even think about proposing a subdomain list. LOL! ;-)
What's the problem with this idea ? It would be only one level above the real host, so for say claranet.de you would have to consider www.claranet.de and xyzzy.claranet.de, but you would ignore www.xyzzy.claranet.de or more.levels.xyzzy.claranet.de
Then if I start to spamvertize my site you catch me without hitting any other user.claranet.de (let alone www.claranet.de)
Assuming that my ISP doesn't neeed weeks to cancel my account after I started to spam the xyzzy entry will expire soon.
Yes, we've thought about this and decided to go with the base domain for several reasons.
1. Spammers use randomized subdomains on many levels above the third or fourth. It would be impossible and also meaningless in many cases to try to capture all of those levels, given the common randomization.
2. Some of the randomized subdomains are unique to a particular spam or batch of spams, therefore logging each unique one both notifies the spammer that their spam got through (thus confirming the recipient address for them) and means that we don't necessarily catch their registered domain.
3. It doesn't focus on what we're trying to go after: the many freshly registered "disposable" spam domains.
4. If a hosting company is legitimate, they will kick out any spammers using subdomains under their parent domain. If they don't then we can begin to consider the hosting company spam-friendly. Most hosting companies do not tolerate subdomain spammers, and that is reflected in the lack of spams we see with subdomains of legitimate domains.
Hopefully some of those will seem at least somewhat reasonable.
It's about time ICANN cracked down on rogue registrars.
I'll believe it when I see it. These registrars pay ICANN's budget, don't they ?
Eventually the registrars should be held accountable for hosting spammer domains. It's good to at least see some movement in that direction.
There will always be disagreement about that optimization point. That is natural. (It's also a PITA.)
Sometimes your criteria appear to be a bit obscure for me. Of course some people may love a "joke of the day" mail - that's okay, if they like it they won't report it as spam.
But others don't like any unsolicited jokes, and they would report it as spam. In that case the joke-of-the-day site _is_ spamming, and it's okay to list them. Even if they also have some real fans with a "legit" interest in their joke of the day. In that case you can't avoid a collateral damage, whatever you do. Bye, Frank
Yes, collateral damage is easily avoided. Don't list them.
Should we ***block everyone else's use*** of the Joke of the day domain? I don't think so.
hmmm.... can't we treat these like we treat com.ar tld? or co.uk? Like Frank said, just checking the subdomain one more level up for these guys. I don't see the harm in that. Or am I missing something again?
I think what Jeff meant was another SURBL list entirely ;) No, I think we have enough as well.
--Chris
See above. There are specific and good reasons why we did not list subdomains of registered domains. No solution is perfect but this one seems to fit the data the best. We actually debated this already, very early in the design process.
Jeff C.
On Friday, September 10, 2004, 5:57:03 PM, Jeff Chan wrote:
From: Frank Ellermann [mailto:nobody@xyzzy.claranet.de]
Sometimes your criteria appear to be a bit obscure for me. Of course some people may love a "joke of the day" mail - that's okay, if they like it they won't report it as spam.
But others don't like any unsolicited jokes, and they would report it as spam. In that case the joke-of-the-day site _is_ spamming, and it's okay to list them. Even if they also have some real fans with a "legit" interest in their joke of the day. In that case you can't avoid a collateral damage, whatever you do. Bye, Frank
Yes, collateral damage is easily avoided. Don't list them.
Should we ***block everyone else's use*** of the Joke of the day domain? I don't think so.
Remember, the goal is to include domains that *only appear in spams*, and to exclude domains that appear in hams. I think that's very clear and simple, not at all obscure. :-)
If we include every domain that anyone has ever considered spam, our data will be too full of false positives for other people to use it. It will basically be useless.
But if we include domains that everyone agrees is spam, such as the pill spammers, warez spammers, etc. then we will have data everyone can use safely.
I hope this is somewhat clear.
Jeff C.
Jeff Chan wrote:
- Spammers use randomized subdomains on many levels above the third or fourth. It would be impossible and also meaningless in many cases to try to capture all of those levels, given the common randomization.
ACK. At the moment we're discussing one level above the base.
- It doesn't focus on what we're trying to go after: the many freshly registered "disposable" spam domains.
The idea is to identify spam by spamvertized URLs. For some time subdomains of free hosters were very popular. That's why I recognize something like "tripod.cl" or "wanadoo.es".
- If a hosting company is legitimate, they will kick out any spammers using subdomains under their parent domain.
Some hosters needed a clue by four. Did I mention tripod.cl ? Or terra.es ? At the moment new domains are state of the art (if spamming is an art), but that will change.
[joke-of-the-domain spam]
Yes, collateral damage is easily avoided. Don't list them.
That _is_ a collateral damage for the recipients of this spam, those who never solicited it and don't want it. If you refuse to list spammers only because some other users might exist who want this crap, then you hurt all users who don't want it.
And vice versa. In that conflict of interests it's not the job of SURBL to protect spammers, but to protect the victims.
Should we ***block everyone else's use*** of the Joke of the day domain?
If this joke-of-the-day is reported often enough via SpamCop as spam, then it should be listed in SC.surbl.org. Otherwise you would censor the SC input data for personal reasons, and that would be wrong.
You should only play god if you're absolutely sure that SC and the SC users screwed up (and this will happen, the spammers try it again and again). SC is only a script, it can't think.
Remember, the goal is to include domains that *only appear in spams*, and to exclude domains that appear in hams. I think that's very clear and simple, not at all obscure. :-)
The goal for SC.surbl.org is to list spamvertized domains, and to identify spam based on the listed domains. It's perfectly neutral, not "some users really want a mortgage from this bank" or similar excuses.
If we include every domain that anyone has ever considered spam, our data will be too full of false positives for other people to use it.
That's why you have technical rules for the SC input data, it's not "anyone", but substantiated facts reflecting SC reports.
It would be a lie if you exclude spamvertized domains for only personal reasons. Sometimes "legit" companies really are so stupid to spamvertize their own domain directly, and then they should be listed if the required number of SC users says so.
Bye, Frank
(1) Interesting article on phishing spam here: http://www.eweek.com/article2/0,1759,1644840,00.asp
(2) BTW - Do we now have a live data feed into PH from antiphishing.org ??
Thanks,
Rob McEwen
On Monday, September 13, 2004, 1:20:15 PM, Rob McEwen wrote:
(1) Interesting article on phishing spam here: http://www.eweek.com/article2/0,1759,1644840,00.asp
(2) BTW - Do we now have a live data feed into PH from antiphishing.org ??
We are talking with them right now about exactly that subject. I'm hopeful It may happen after they meet this month.
Jeff C.
On Monday, September 13, 2004, 7:45:44 AM, Frank Ellermann wrote:
Jeff Chan wrote:
- If a hosting company is legitimate, they will kick out any spammers using subdomains under their parent domain.
Some hosters needed a clue by four. Did I mention tripod.cl ? Or terra.es ? At the moment new domains are state of the art (if spamming is an art), but that will change.
Terra.es definitely kicks spammers. I've seen them do it.
[joke-of-the-domain spam]
Yes, collateral damage is easily avoided. Don't list them.
That _is_ a collateral damage for the recipients of this spam, those who never solicited it and don't want it. If you refuse to list spammers only because some other users might exist who want this crap, then you hurt all users who don't want it.
And vice versa. In that conflict of interests it's not the job of SURBL to protect spammers, but to protect the victims.
People who's legitimate messages are blocked due to over-inclusive blocklists are also victims. But they are *victims caused by **our** actions*, not by the spammers. We should NOT ***CAUSE*** VICTIMS.
We need to be like doctors: do no harm. If we let through a few spams, that's much better than blocking someone's legitimate mail.
I think many people do not understand that, and that is a definite problem.
If we cause FPs, we are doing more harm than good.
It's better to let a couple spams through than for our tools to ***cause*** harm to people. We should not **create** victims by having FPs.
Should we ***block everyone else's use*** of the Joke of the day domain?
If this joke-of-the-day is reported often enough via SpamCop as spam, then it should be listed in SC.surbl.org. Otherwise you would censor the SC input data for personal reasons, and that would be wrong.
You should only play god if you're absolutely sure that SC and the SC users screwed up (and this will happen, the spammers try it again and again). SC is only a script, it can't think.
If the SC users are trying to list messages as spams that other people consider hams, then they have screwed up. We reserve the right to correct their mistake. Mistakes do happen occasionally.
Remember, the goal is to include domains that *only appear in spams*, and to exclude domains that appear in hams. I think that's very clear and simple, not at all obscure. :-)
The goal for SC.surbl.org is to list spamvertized domains, and to identify spam based on the listed domains. It's perfectly neutral, not "some users really want a mortgage from this bank" or similar excuses.
We are trying to make lists that do not have false positives. A list that has no false positives will probably miss a few spams. It's MUCH better to miss a few spams than to block someone's legitimate mail due to false positives.
Real banks don't send mortgage spams. Real banks don't use zombies. Have you ever gotten a Viagra spam from Pfizer? I haven't.
If we include every domain that anyone has ever considered spam, our data will be too full of false positives for other people to use it.
That's why you have technical rules for the SC input data, it's not "anyone", but substantiated facts reflecting SC reports.
SC users are sometimes wrong. They are not perfect. They sometimes try to report sites that have legitimate uses.
It would be a lie if you exclude spamvertized domains for only personal reasons. Sometimes "legit" companies really are so stupid to spamvertize their own domain directly, and then they should be listed if the required number of SC users says so.
Bye, Frank
It's not "personal reasons" if other people use a domain legitimately. That's highly impersonal reasons. We don't need to know any of the legitimate users, to want to protect them from incorrect blocking.
Lots of spams come out of topica or lyris. Should we block them? Of course not. The legitimate uses outweigh any spams abusers can send out before they are shut down. Yes they are a source of some spam, but blocking them would cause more harm than good.
I think if you're not understanding this point, there's not much reason to debate it further.
Jeff C.
-
Chris Santerre -
Frank Ellermann -
Jeff Chan -
Rob McEwen