[SURBL-Discuss] "Unconfirmed" list proof-of-concept

Rob McEwen (PowerView Systems) rob at powerviewsystems.com
Fri Sep 3 14:22:17 CEST 2004


> If Ryan (or someone else) wants to take submissions directly, for 
> different lists, fine, but do not use the mailing list for this purpose, 
> the volume is high enough as it is now.
> 
> Again, this mail is not trying to upset anyone, i think most of you will
> understand our point of view, if not we would be willing to point that out
> offlist.

Even thought I vote FOR unconfirmed.surbl.org, I'd rather wait and let this issue work itself out than rush ahead and, as Jeff put it, "divide our efforts".

Therefore, I have some constructive suggestions for how to unify our efforts in the meantime towards things I think we can all agree on which will only help achieve ALL of our goals, **even the ones that we disagree on :)** (I'll explain how...)

FIRST:

While we are waiting to see (for sure) what happens with unconfirmed.surbl.org, there are other ways to do greylisting. Please allow me to share some of the things that I do. (forgive me that some of these are unconventional and are not "by the book"... no lectures please).

**I convert domains to IP addresses both in the body and headers and check these IP addresses and other IP address in the message against SpamHaus (they even suggest doing this in their literature... at least to some extent).

**I check header URIs against SURBL (again, no lectures please... I do this more for auditing than for actual blocking)

NOTE...I have actually found that these first two generate a high rate of spam with few false positives.... but there ARE enough false positives to keep me from using this for blocking in a "set it and forget it" mode, but... a high amount of spam "hits" nevertheless.

**After my SpamHaus/DSBL MTA blocking is done, and after my linguistic filter & SURBL filtering is done... I then check the sending IP of each message still not blocked against both psbl.surriel.com and bl.spamcop.net . A lot of legitimate messages get "flagged" here... but still a rich auditing pool. This helps me quickly find much of what is getting past my filter without having to constantly dig through my client's e-mail.

...ALSO, in all these cases, I create bypass lists for each level of checking to prevent false "hits" on each of these checks so that I'm not constantly looking at the same stuff over and over again. I also blacklist obvious URIs in my linguistic filter that I deem not possibly ever legitimate. (I should probably be submitting these to SURBL? ... I have some follow questions on that process for another tread.)

Looking at the data and what actually happens... my SpamHaus technique **IS** almost like what I think that people are looking for in unconfirmed.surbl.org

SECOND:

5 days ago, I had a suggestion regarding integrating tracking of these Greymarketer organizations into the upcoming formal SURBL submission and tracking system we've talked about. The basic idea was to find a way to put pressure on them, generate an awareness of these groups, and try to help each other develop tools for blocking their spams without blocking the legitimate messages.

Changing the subject slightly, another idea behind this was to work together to develop strategies for blocking spam from those proliferate spammers who often beat SURBL for a few hours or days with their new domains. For example, recall that "pesky porn spam" request I made last month... one of the guys on this list (forgot his name) gave me some suggestions and I develop a rule in my filter which catches ALL of these with zero chance of false positives.

FINALLY:

Overall, I suggest that there are plenty of things we can work on... even "out of the box" things... that we can all agree on. I suggest we work on these and shelve "unconfirmed.surbl.org" in the meantime. A good compromise is to take my suggestion and integrate tracking of things that would be good candidates for the UC list a a formal category in the format SURBL tracking system that we all agreed should be created. That way, if we later DO formally decide to do the UC list, the pro-UC folks will know that they are already further along in that task than we are now.

Thoughts?


BELOW IS MY EARLIER MESSAGE FROM ABOUT A WEEK AGO THAT I REFERRED TO:

I think that there are two areas which present particular challenges.

(1) e-mail marketers who play it "both ways"... thus making it hard to use
SURBL to catch their bad behavior without blocking legitimate mail

...AND...

(2) savy spammers who manage to get significant amounts through in the first
few minutes/hours BEFORE getting blocked by SURBL... in particular, the ones
who already use the best strategies to get around all other types of
filtering.

The quicker TTLs is helping with the savvy spammers. Also, I recall something
about a newer version of SURBL which will use some kind of tracking to trace
new domains back to older ones in order to attributing new spam to known and
confirmed spammers so that they would stay "attached" to their previous bad
records in order to blacklist them faster. What ever became of this? (Did I
explain this correctly?)

Anyway... even when these are done, we will STILL have some problems with
the most savvy spammers.

Also, I think that a lot of people fear that, as we work towards eliminating
the rest of the FPs, more and more spam from these e-mail marketers who play
it "both ways" will get through and the overall catch rate for SURBL may
drop by 10 or 20 percent (or whatever).

I'm willing to live with that... (gulp!)

BUT... I think that it would be great to integrate into a formal tracking
system a way to categorize URIs into either or both of these groups.
("SavySpams" and "GrayMarketer" ...or whatever) That way, we can use this
data to help us form better "rules" in our linguistic/heuristic filters. The
idea being that, at this point, the amount of spam that is getting through
is much more focused than a large general pool of spam. This more narrow
focus should give us the tools to close any loopholes that SURBL might not
catch.

I would also suggest that if a message's server address is already blocked
by BOTH list.dsbl.org AND sbl-xbl.spamhaus.org, then it shouldn't be added
to this particular list for the sake of keeping the list focused. It seems
that, whatever the disagreements about RLBs are, I think that EVERYONE would
agree with this particular standard as being a reliable (yet FP safe and
conservative) standard for RBL blocking.

I envision a "Gray page" which would list the top 10 offenders of
Graymarketers who are bad enough to be mentioned, but not bad enough to get
listed by SURBL and the top 10 "savvy spammers" who are known to periodically
(abet temporarily) beat SURBL with their new domains. Subsequent offenders
could be listed on following pages after the top 10 for each of these two
categories. Each listing would include a link to more info about this
spammer or series of spam. This more info page would also included samples
of spam that hit real spamtraps (made anonymous), and, for the gray
marketers, samples of legitimate mail with that particular URI.

FP-safe rules would also be suggested...

NOW... who would have the time to get all this together??? :)

Rob McEwen


More information about the Discuss mailing list