On Wednesday, April 6, 2005, 12:55:10 PM, Frank Ellermann wrote:

If you have an idea which doesn't fit into multi.* you could still use the existing infrastruture in a cs.surbl.org zone, in theory. In practice you apparently came to the conclusion that that's not what you want

For SURBLs we don't want to have false positives or harm innocent bystanders. Other lists may have other policies. For example Spamhaus and other RBLs deliberately list networks that are partially used by spammers but also have non-spammers using them. Those kinds of policies may be fine for those RBLs, but they can cause "collateral damage" that SURBL's listing policy tries to avoid.

The general design of SURBLs is very specific to domains that actually appear in spams, so deliberately inflicting collateral damage should not be necessary. Because of that specificity, we don't need to harm innocent bystanders in order to block most spam sent by the high-volume, zombie-using, spam gangs.

A list of legitimate companies that sometimes send spam, which is what Chris means when he says a grey list of "mainsleazers" would definitely be one that hits some hams and would cause some collateral damage. We would not want to do that with SURBLs, at least not if we keep our current policies.

One of the reasons SURBLs have been very useful to many people is because of these policies. So we're probably not going to change them. That said, there's probably room for other lists for folks who want to be more aggressive about mail filtering, but we're probably never going to do that with SURBLs. Our goal with SURBLs is to make a list that can be "set and forget" and used by large ISPs, telcos, etc. without hitting much ham.

Part of the difficulty is that different people have different ideas about what consitutes spam. Some people think anyone who has ever sent one spam is forever a spammer, even if it's your Aunt Millie, Microsoft or General Motors. They may or may not be spammers, but more importantly their domains appear in hams that ordinary people probably want to get so we probably don't want to blacklist them. On the other hand most people probably don't want to get spams sent by Alan Ralsky's zombies, or Scott Ricther's junk mail, etc. So it somewhat depends on your definition of spam and how much ham you want to block.

Jeff C. -- "If it appears in hams, then don't list it."

On Wednesday, April 6, 2005, 10:37:47 PM, SM wrote:

The discussion here is more about what to do about bystanders who are used to send out spam.

No, because SURBLs are not sender lists, but I see your point below.

Most antispam techniques are quite effective at the beginning. That rate drops as spammers come up with ways to circumvent the technique. SURBL has introduced a grey area where redirectors are used to send out a url. Given SURBL's listing policy, these domains won't be listed as it causes false positives. There is no incentive for domains which fall in the grey area to fix their redirectors.

Sure there is. Do they like having the servers hit with millions of spam redirections? Those cost resources and don't get them legitimate click throughs or whatever. It's just abuse of their systems by spammers.

Many of the organizations that have been contacted about their open redirectors being used in spams have attempted to fix them with varying degrees of success. But most realize it's abuse and wasting their resources and at least try to do something about it.

Jeff C. -- "If it appears in hams, then don't list it."

Jeff Chan wrote:

For SURBLs we don't want to have false positives or harm innocent bystanders.

It would be utter dubious to mix completely incompatible strategies in one list like multi.surbl.org (a bad example is SORBS 127.0.0.6, whois.RFCI is also not beyond doubt if you don't check the exact code).

But different uncombined lists can be completely different, like say wadb.isipp is not iadb.isipp

we don't need to harm innocent bystanders in order to block most spam sent by the high-volume, zombie-using, spam gangs.

At the moment. They will adapt and try new tricks like using redirectors or similar bridge pages on free and cheap hosters.

Part of the difficulty is that different people have different ideas about what consitutes spam.

They also have different ideas about what causes harm to IBs: There's almost no convincing reason to use redirectors like Google in e-mail, unless it's spam or a pay per click fraud.

Bye, Frank

On Thursday, April 7, 2005, 12:36:49 PM, Patrik Nilsson wrote:

At 20:51 2005-04-06 -0700, Jeff Chan wrote:

...

One of the reasons SURBLs have been very useful to many people is because of these policies. So we're probably not going to change them. That said, there's probably room for other lists for folks who want to be more aggressive about mail filtering, but we're probably never going to do that with SURBLs.

It's not just about lists for folks who want to be more aggressive about spam filtering, it's also about people who want to use surbl-type lists to *score*, rather than block, spam.

I am not sure when it happened, but SURBLs are apparently now explicitly expected to be used for *blocking* email.

That was always a stated goal from the beginning: to have a telco-grade list that could be used for outright "set it and forget it" blocking at the MTA level.

That may be an unreachable goal, but the fact that (some) SURBLs get high scores in SpamAssassin is an indication that some may approach that goal.

Separate lists primarily used for scoring could be more aggressive in their listing policies, without necessarily ending up blocking more FPs.

To some extent we already have that. sc.surbl.org gets a 3.9 score in SA. ws.surbl.org, a more aggressive list, gets 0.5 .

The current policy for SURBLs, currently being the only major public uri list, encourages binary "block or not" implementations that expect surbl type lists to be used in that capacity. That's bad, as it unnecessarily limits the potential for url checking.

score URIBL_AB_SURBL 0 2.007 0 0.417 score URIBL_OB_SURBL 0 1.996 0 3.213 score URIBL_PH_SURBL 0 0.839 0 2.000 score URIBL_SC_SURBL 0 3.897 0 4.263 score URIBL_WS_SURBL 0 0.539 0 1.462

The differing scores say otherwise.

Having alternative lists with other policies could hopefully make implementors think twice about how they implement url checking against surbl type lists.

And no, this is not a problem caused by SURBL - I can understand why the policy is like it is considering what it apparently wants to do - but I really believe we should encourage other surbl type lists to avoid limiting the use of url checking in anti-spam software.

Patrik

Anyone is free to make whatever lists they want with whatever policies they want. We're not going to change the policies on SURBLs however. Many people are using SURBLs because of the policies we have and the results they get by using them.

We have no plans to change what we're doing, and we have the support of many active contributors and data sources. I don't see that changing. Nor are we changing our listing policies.

SURBLs will continue to be around and hopefully will have improved FP rates and better spam detection rates too.

Jeff C. -- "If it appears in hams, then don't list it."