- Discuss - Mailman 3

False Positives
by Mariano Absatz 06 Sep '04

06 Sep '04

Hi, browsing my autodetected spam folder, I came across a couple of messages from Randy Cassingham's 'This is True' newsletter (see http://thisistrue.com) that hit on SURBL lists.. Randy uses plain text advertising within is bulletin but he's not a spammer... in fact he wrote and maintains a 'spam primer' page at http://www.spamprimer.com/ The domains that hit this where: REMOVETHIS-JumboJoke.com and REMOVETHIS-BonzerSites.com (in outblaze's list) and REMOVETHIS-HealthYouDeserve.com (in Bill's list). -- Mariano Absatz - El Baby el (dot) baby (AT) gmail (dot) com el (punto) baby (ARROBA:@) gmail (punto) com

2 5

fp hsc news dot com
by Jose Marcio Martins da Cruz 06 Sep '04

06 Sep '04

Hello, "hsc - news dot com" is listed at ws and multi.surbl. This a very serious french security consulting group. Surely a false positive. Can you remove it ? They run a list but only subscription is **only** done when **explicitely** asked by recipients - I know as I'm one of their subscribers. Best regards, Jose-Marcio -- --------------------------------------------------------------- Jose Marcio MARTINS DA CRUZ Tel. :(33) 01.40.51.93.41 Ecole des Mines de Paris http://j-chkmail.ensmp.fr 60, bd Saint Michel http://www.ensmp.fr/~martins 75272 - PARIS CEDEX 06 mailto:Jose-Marcio.Martins@ensmp.fr

5 8

Domain age
by Ryan Thompson 06 Sep '04

06 Sep '04

Hi all, I added some experimental code to GetURI to automatically determine the age of a domain, which works for about 99.2% of the domains I've seen in SURBL, and, with a little bit of gaussian math, the results are fricken' *amazing* for classification! Unfortunately, I need a way to be able to do this without violating registry whois terms of service, because they don't allow automated queries, "except as reasonably required to register and update domains" or somesuch... Any ideas? - Ryan -- Ryan Thompson <ryan(a)sasknow.com> SaskNow Technologies - http://www.sasknow.com 901-1st Avenue North - Saskatoon, SK - S7K 1Y4 Tel: 306-664-3600 Fax: 306-244-7037 Saskatoon Toll-Free: 877-727-5669 (877-SASKNOW) North America

3 2

Ham corpora needed
by Jeff Chan 06 Sep '04

06 Sep '04

In order to reduce false positives in the SURBL data, we would like to have access to ham corpora. Does anyone know of any public ham copora, including just the URI domain names from the hams? Or is there anyone who would be willing to run our URI domain lists against their ham? Does anyone know if messages from the Enron corpus have been categorized for ham and spam? http://www-2.cs.cmu.edu/~enron/ Thanks in advance for any suggestions, comments, thoughts.... Jeff C. -- Jeff Chan mailto:jeffc@surbl.org http://www.surbl.org/

3 12

whitelisted x10.com
by Jeff Chan 06 Sep '04

06 Sep '04

I've whitelisted x10.com. They are a frequent FP since they "advertise" so much to their own customers. I'm surprised it was not whitelisted earlier. x10.com is probably significant contributor to the ham scores. Jeff C.

1 0

Setting SpamAssassin scores for SURBL lists
by Jeff Chan 05 Sep '04

05 Sep '04

Eric Kolve and I were looking at how to best set the default SpamCopURI scores for the various SURBL lists and at first we tried looking at the SpamAssassin 3.0 perceptron-generated scores as a possible guide: > http://spamassassin.apache.org/full/3.0.x/dist/rules/50_scores.cf > > # The following block of scores were generated using the mass-checking > # scripts, and a perceptron to determine the optimum scores which > # resulted in minimum false positives or negatives. The scores are > # weighted to produce roughly 1 false positive in 2500 non-spam messages > # using the default threshold of 5.0. > score URIBL_AB_SURBL 0 2.007 0 0.417 > score URIBL_OB_SURBL 0 1.996 0 3.213 > score URIBL_PH_SURBL 0 0.839 0 2.000 > score URIBL_SC_SURBL 0 3.897 0 4.263 > score URIBL_WS_SURBL 0 0.539 0 1.462 I was trying to figure out what the different score columns meant, to which Theo Van Dinter cited: > $ perldoc Mail::SpamAssassin::Conf > [...] > If four valid scores are listed, then the score that is used > depends on how SpamAssassin is being used. The first score is used > when both Bayes and network tests are disabled (score set 0). The > second score is used when Bayes is disabled, but network tests are > enabled (score set 1). The third score is used when Bayes is > enabled and network tests are disabled (score set 2). The fourth > score is used when Bayes is enabled and network tests are enabled > (score set 3). We wondered if we could somehow use those scores with SpamCopURI and were unable to come up with a good answer. Theo suggested looking at Spam versus ham rates as a good way to set scores, to which I mentioned: > We have these test results from Justin from 25 June: > > OVERALL% SPAM% HAM% S/O RANK SCORE NAME > 121405 22516 98889 0.185 0.00 0.00 (all messages) > 100.000 18.5462 81.4538 0.185 0.00 0.00 (all messages as %) > 13.453 70.3766 0.4925 0.993 1.00 1.00 SURBL_WS > 3.807 20.3811 0.0334 0.998 0.50 1.00 SURBL_SC > 2.650 14.2565 0.0071 1.000 0.50 1.00 SURBL_AB > 0.019 0.0933 0.0020 0.979 0.50 1.00 SURBL_PH > 12.624 67.6275 0.1001 0.999 0.50 1.00 SURBL_OB > > which shows a pretty high FP rate for WS, less for the others. > Do you happen to have access to any more recent corpus check data > like this? Could be useful to have another snapshot for a more > complete picture. Which was followed up with more data and discussion: > On Saturday, September 4, 2004, 10:13:11 PM, Theo Dinter wrote: >> high spam + low ham is good from an FP standpoint, but having a "significant" >> (for your definition thereof) ham hitrate means the score shouldn't be too >> high. My handwaving scores would be something like: [Theo's wild guess scores for Justin's June data: -- Jeff C.] >> WS 1.2 >> SC 2.5 >> AB 3.5 >> OB 1.8 Theo then gave some of his own stats on a couple different corpora: >> OVERALL% SPAM% HAM% S/O RANK SCORE NAME >> 416072 365031 51041 0.877 0.00 0.00 (all messages) >> 100.000 87.7327 12.2673 0.877 0.00 0.00 (all messages as %) >> set1 30.923 35.2466 0.0000 1.000 0.99 0.00 URIBL_SC_SURBL >> set1 72.231 82.3273 0.0274 1.000 0.98 1.00 URIBL_OB_SURBL >> set1 19.375 22.0847 0.0000 1.000 0.98 1.00 URIBL_AB_SURBL >> set1 74.883 85.2939 0.4310 0.995 0.74 0.00 URIBL_WS_SURBL >> set1 0.001 0.0000 0.0059 0.000 0.48 0.00 URIBL_PH_SURBL > >> OVERALL% SPAM% HAM% S/O RANK SCORE NAME >> 119215 67094 52121 0.563 0.00 0.00 (all messages) >> 100.000 56.2798 43.7202 0.563 0.00 0.00 (all messages as %) >> set3 39.217 69.6605 0.0288 1.000 0.98 1.00 URIBL_OB_SURBL >> set3 10.340 18.3727 0.0000 1.000 0.97 0.00 URIBL_SC_SURBL >> set3 5.998 10.6582 0.0000 1.000 0.94 1.00 URIBL_AB_SURBL >> set3 42.730 75.5522 0.4797 0.994 0.73 0.00 URIBL_WS_SURBL >> set3 0.008 0.0089 0.0058 0.608 0.49 0.00 URIBL_PH_SURBL > >> so for these results, I'd probably do something like: > >> WS 1.3 >> SC 4.0 >> AB 3.0 >> OB 2.2 > >> since the hit rates and S/O are a bit higher for me, related to the fact I ran >> more spam through than Justin did. To which I added: > Those final scores look like an excellent fit to the data to me. and: > Also while the PH spam hit rate [from Justin's stats] is low, > the data is of hand checked phishing scams, which deserve to be > blocked due to their potential danger and damage. > > Therefore I would tend to give PH a medium-high score like > 3 to 5. So we'll probably adjust the default scores on SpamCopURI to something like: WS 1.3 SC 4.0 AB 3.0 OB 2.2 PH 4.5 and we recommend SpamCopURI users do likewise. Please be sure to use the latest version of SpamCopURI with multi.surbl.org: http://sourceforge.net/projects/spamcopuri/ http://search.cpan.org/dist/Mail-SpamAssassin-SpamCopURI/ One thing stood out for me is that the FP rate (ham%) for ws.surbl.org is way too high at about 0.45 to 0.5% across multiple corpora. That FP rate needs to be reduced for WS to be more fully useful. I think Chris or maybe Raymond suggested that they had a way to reduce FPs in WS further. If so, ***please*** try to apply it. We need to get the FPs to be much less than 0.5%. The other lists have FP rates 5 to 50 times lower. Basically the higher the FP rate, the less useful a list is. Does anyone have other corpus stats to share, in particular FP rates? Jeff C. -- Jeff Chan mailto:jeffc@surbl.org http://www.surbl.org/

3 7

Fwd: Re: [SURBL-Discuss] checking plain domains in message bodies against SURBLs reportedly effective
by Jeff Chan 05 Sep '04

05 Sep '04

This is a forwarded message From: Theo Van Dinter <felicity(a)kluge.net> To: SURBL Discussion list <discuss(a)lists.surbl.org>, SpamAssassin Developers <spamassassin-dev(a)incubator.apache.org> Date: Saturday, September 4, 2004, 10:36:53 AM Subject: [SURBL-Discuss] checking plain domains in message bodies against SURBLs reportedly effective ===8<==============Original message text=============== On Sat, Sep 04, 2004 at 10:45:44AM -0600, Ryan Thompson wrote: > Yep. Good idea, overall. There are a few gotchas: > > TLD extensions sometimes map file extensions. We might have to whitelist > command.com, and the entire country of Poland. :-) > > Since the domain is in plain text and doesn't contain a protocol or > subdomain (i.e., 'www'), I haven't yet seen a mail client that will > display it as a clickable URL. This is generally the tact we're taking in SpamAssassin -- if a general MUA doesn't display it as a link, then we don't consider it an URL. Another issue for the generic domains thing is performance -- lots of messages have lots of things like could potentially look like a domain, and querying for them all adds a bit of a load on the client and the server. For instance: /\b([a-zA-Z0-9_.-]{1,256}\.[a-zA-Z]{2,6})\b/ in theory (I haven't tested it), will grab anything that looks like a generic domain name in text. If you check that list against a list of valid TLDs, you'd probably end up with a decent list, but you'd hit the top issue quoted above where "Go take a look at command.com" isn't clear if it's an URL or a filename. -- Randomly Generated Tagline: "Brevity is the soul of lingerie." - Dorothy Parker ===8<===========End of original message text===========

1 0

[SURBL-Discuss] checking plain domains in message bodies against SURBLs reportedly effective
by Jeff Chan 04 Sep '04

04 Sep '04

Randy Brukardt of rrsoftware.com mentioned that checking plain domains occurring in message bodies against SURBLs was pretty productive. (E.g., look for domain.com in addition to www.domain.com or http://www.domain.com). Perhaps this could be something interesting to at least try experimentally or to think about. Jeff C. -- Jeff Chan mailto:jeffc@surbl.org http://www.surbl.org/

3 5

"safelists"
by Joe Wein 04 Sep '04

04 Sep '04

Today I processed a pile of 11,000 spams donated by Raymond, as I'm gradually gearing up for large-scale spamtrap processing. I ended up with many most obvious spam domains, but an even bigger pile of domains that aren't quite as suspicious. Many are from last year and the name server is not listed by SBL. I don't want to add any grey domains to a black pile. While investigating many of them to determine if they deserved listing, I repeatedly came across the term "safelist". They seem to be some kind of opt-in lists. What is the term supposed to mean exactly? Joe -- http://www.joewein.de/sw/jwSpamSpy/

2 1

Re: [SURBL-Announce] PROBLEM: WS.SURBL.ORG frozen
by Jeff Chan 04 Sep '04

04 Sep '04

On Friday, September 3, 2004, 3:10:29 PM, Raymond Dijkxhoorn wrote: > Currently we are running with a somehow frozen ws.surbl.org list. We are > experiencing hardware trouble with one of the SURBL machines. New updates > will be processed, but most likely activated after we restore full > functionality. > The main SURBL site is not afffected, its only related to the WS updates. > We are working hard to get the processing box back online. We made a temporary workaround to get updates from Raymond and others directly into ws.surbl.org until the other server comes back. Once the other server is working again we will undo that workaround. So ws is now getting updated with at least some of the new data. Jeff C.

2 1

whitelist: cooleremail.com and icebase.net?
by Jeff Chan 04 Sep '04

04 Sep '04

cooleremail.com is apparently on WS and asking to be whitelisted. Does anyone know anything about them or why they are on WS? They seem to have reasonable antispam policies: http://www.cooleremail.com/aboutus_spam.shtml Jeff C.

3 6

Perfect example!
by Chris Santerre 04 Sep '04

04 Sep '04

tourisme-montreal.org Just sent a spam to our president. However it has legit uses. Screw them! Here is a nice fresh one for UC :) --Chris

3 4

RFC: Public false positve reporting form
by Jeff Chan 04 Sep '04

04 Sep '04

How does anyone feel about a public false positive reporting form? Such a form would include the reporting party, the domain(s) or IPs, reasons why it's a false positive, etc. It could be processed publically or privately in a bug tracking system. It would be hand checked by some of our experienced spam fighters. The results could be published or simply aggregated without public announcement into the overall whitelist. Is this a good or bad idea? What level of publication should there be? What forms of proof (if any) does anyone like? Other comments? Jeff C.

4 4

RE: [SURBL-Discuss] Proposing a greylist
by Chris Santerre 03 Sep '04

03 Sep '04

It has been moved off list. See Ryan's post. If you would like to be involved, contact him. --Chris (wishes this thread to go away now.) >-----Original Message----- >From: Bret Miller [mailto:bret.miller@wcg.org] >Sent: Friday, September 03, 2004 5:00 PM >To: SURBL Discussion list >Subject: RE: [SURBL-Discuss] Proposing a greylist > > >To further this idea, comparing it to DNSBLs is appropriate. >There are a >lot of DSNBLs you can use. Many agree that sbl-xbl spamhaus lists are >high quality with no (or almost no) false positives. For that reason, a >lot of us use them to block e-mail. > >Bl.spamcop.net is another DNSBL, and while it's a fine list, >the FP rate >is much too high for me to block e-mail on it alone. However, it's >highly useful in SpamAssassin, putting many messages over the >spam score >threshold. > >While the SURBL staff has decided that they would rather not "divide >their efforts" to support a project like this, it certainly >doesn't mean >that it shouldn't be done be someone else. At that point, the staff is >right to ask that the discussion about its details be moved off this >list. Perhaps the better place to discuss this is the SA >general list as >they are more open to just about any new project that helps SA detect >spam accurately. > >I know if I relied totally on any one method of filtering >spam, a lot of >spam would be getting through that isn't. It's because the >comination of >content and structure analysis with bayesian analysis DNSBL and URIBL >all contributes to the end result. > >Bret > > > > >_______________________________________________ >Discuss mailing list >Discuss(a)lists.surbl.org >http://lists.surbl.org/mailman/listinfo/discuss >

1 0

Proposing a greylist
by Chris Santerre 03 Sep '04

03 Sep '04

I am officially proposing a greylist surbl. We are going to see more and more of this stuff. We might as well deal with it now. I'm suggesting a greylist for all spammers that ride that line. Like the euniverse junk we have been talking about. 1)We DO NOT include it in multi. 2)We SCREAM to the world that it WILL hit some legit, and that only hard liners should use. 3)We DON'T remove domains unless they go completely black, or have no NANAS hits for 3-4 months. 4)See number 2 again. 5)We tell people it is completely optional and to see number 2. I predict it would be used more for personal emails. IT also gives us an in between mechanism. Rather then list or no list. We get a grey list we desperately need. THoughts? Chris Santerre System Admin and SARE Ninja http://www.rulesemporium.com http://www.surbl.org 'It is not the strongest of the species that survives, not the most intelligent, but the one most responsive to change.' Charles Darwin

12 31

RE: RE: [SURBL-Discuss] "Unconfirmed" list proof-of-concept
by Chris Santerre 03 Sep '04

03 Sep '04

>-----Original Message----- >From: Rob McEwen [mailto:rob@powerviewsystems.com] >Sent: Friday, September 03, 2004 4:04 PM >To: SURBL Discussion list >Subject: Re: RE: [SURBL-Discuss] "Unconfirmed" list proof-of-concept > > >Chris and Ryan... > >I think that ya'll are making a mistake... please reconsider >the advice I gave in my last two e-mails (did you read them?). > >Basically, you would have gotten just about everything you >ever wanted by following my advice (not the Spamhaus part... >but, rather the tracking part) and it would all then be done >with official SURBL-participation and blessings... (did you >yawn during the SpamHaus part of my message and ignore the >rest?)... of course, this assumes that Jeff will warm up to my >suggestion in the first place?? > >But, if your way is inevitable, I just might join you... just >please reconsider in the meantime... and wait to see what >Jeff's final word is on this matter. > Jeff has given his final word on this. Was pretty clear :) And I respect Jeff's wishes. This really isn't that big of a deal. Its going to be completely private until a good data set is made, then made availible for testing by SURBL members if they want it. Just another list, that won't be attached to the SURBL name. Discussions won't be carried out in the SURBL list. Of course if in normal SURBL traffic we find a domain unfit for blacklisting, we may see fit to add to UC. Otherwise for now it will seem like this discusion never took place :) --Chris (You saw nothing.)

1 0

RE: [SURBL-Discuss] "Unconfirmed" list proof-of-concept
by Chris Santerre 03 Sep '04

03 Sep '04

LOL yeah I already sent him an email saying to let us know when it is back up. I'm sure he is ready for an ice cold brew! --Chris (Top posting for fun and profit!) >-----Original Message----- >From: Alex Broens [mailto:surbl@alexb.ch] >Sent: Friday, September 03, 2004 4:03 PM >To: SURBL Discussion list >Subject: Re: [SURBL-Discuss] "Unconfirmed" list proof-of-concept > > >Chris Santerre wrote: >> Look mom! I have another project! :) >> >> I think I'll start an Apache project that attempts to >combine other Apache >> projects! Woot! > >and here's another: You tell me what to do with all those nice >ZeroHour >URIS I'm collecting while poor Bill is sweating? > >Fosters time... > >Alex >

1 0

Re: RE: [SURBL-Discuss] "Unconfirmed" list proof-of-concept
by Rob McEwen 03 Sep '04

03 Sep '04

Chris and Ryan... I think that ya'll are making a mistake... please reconsider the advice I gave in my last two e-mails (did you read them?). Basically, you would have gotten just about everything you ever wanted by following my advice (not the Spamhaus part... but, rather the tracking part) and it would all then be done with official SURBL-participation and blessings... (did you yawn during the SpamHaus part of my message and ignore the rest?)... of course, this assumes that Jeff will warm up to my suggestion in the first place?? But, if your way is inevitable, I just might join you... just please reconsider in the meantime... and wait to see what Jeff's final word is on this matter. Rob McEwen

1 0

RE: [SURBL-Discuss] "Unconfirmed" list proof-of-concept
by Chris Santerre 03 Sep '04

03 Sep '04

>-----Original Message----- >From: Ryan Thompson [mailto:ryan@sasknow.com] >Sent: Friday, September 03, 2004 3:13 PM >To: SURBL Discussion list >Subject: Re: [SURBL-Discuss] "Unconfirmed" list proof-of-concept > > > >All, > >I'm *really* glad to see all of the lively discussion going on here >regarding the grey list / uc idea. That alone, I think, speaks to the >potential of such an idea. However, as Raymond and Jeff have both >pointed out, they're not at all keen on including this in the SURBL >project itself, and want to keep mailing list traffic down. > >I respect that. I also still think there's a lot of merit to the UC >list. I'd like to protect SURBL's interests, and make the new list >happen. So, I'd encourage anyone interested in the grey list / uc idea >to contact me off-list at <ryan(a)sasknow.com> so we can get organized, >move forward, and also respect the wishes of the fine surbl.org folks. +1 here. Look mom! I have another project! :) I think I'll start an Apache project that attempts to combine other Apache projects! Woot! --Chris (The pimp daddy to open source antispam.)

2 1

To MUNGE to not to MUNGE
by Ryan Thompson 03 Sep '04

03 Sep '04

Early mornin' rant: I know the SURBL.org FAQ documents the use of "MUNGED" to munge URIs for discussion. Personally, I think it's silly to do that while sending to this list, especially since, in the same heading, SURBL.org also suggests "It's probably a good idea to use less filtering on your anti-spam mailing list messages, or even to whitelist them". I think that's a much better idea, and everyone here should be doing that anyway, since live spams do often appear on this list. I know I do. The MUNGED is largely unnecessary, takes time to deal with, and (just as it is supposed to) stops automatic URI parsing, and even severely impairs human URI parsing, to the point that I go cross-eyed after spending half an hour reading the finer points on why "someMUNGEDwebsite.com" should be whitelisted. Oh, by the way, SpamAssassin, at least (and all others that I'm aware of), will not even parse body tokens of the form domain.com or 127.0.0.2 as URIs. And, again, "use less filtering on your anti-spam mailing list messages", if you aren't already. I see no reason to MUNGE domains posted to this list. Unless someone has some really compelling evidence to the contrary, could we please curb this silly practice before it really becomes a silly collective habit? :-) - Ryan -- Ryan Thompson <ryan(a)sasknow.com> SaskNow Technologies - http://www.sasknow.com 901-1st Avenue North - Saskatoon, SK - S7K 1Y4 Tel: 306-664-3600 Fax: 306-244-7037 Saskatoon Toll-Free: 877-727-5669 (877-SASKNOW) North America

8 16

Spammer or not: cyberpolice.ru
by Jeff Chan 03 Sep '04

03 Sep '04

cyberpolice.ru just got added to sc.surbl.org. Does anyone know if they are spammers or not? Jeff C.

5 7

"Unconfirmed" list proof-of-concept
by Ryan Thompson 03 Sep '04

03 Sep '04

As some of you may have read, I have set up an "unconfirmed" ("uc") list (or grey list), ready for submissions. It seems to me that there are at least a few active members of this list interested in giving this a shot. Currently, I am the only one with access to the data, but this can easily be changed to include a small number of trusted individuals with an interest in helping out. Here's how it works (and I'm putting this out as a starting point for discussion, not the gospel!) 1. When classifying domains for SURBL, continue to employ the same level of effort you do to find all domains worthy of blacklisting, and submit those to the usual sources. 2. For the domains that don't fit black lists or white lists, they *may* be candidates for uc. However, we only want to list domains in uc that are highly spammy, with only very small potential for legitimate use. HOW TO SUBMIT: a) Chris Santerre has suggested domains for inclusion in uc be posted to the SURBL mailing list, and only be included after receiving a total of three votes. I'll go with that, and suggest using a Subject like "UC: domain.com" for easy spotting. b) I'd also encourage submitters to email the submission address, uc(a)sasknow.com, with private submissions of one or more domains, one per line. I'll handle private submissions in the same way as mailing list submissions; if I get three separate private submissions for the same domain (or maybe two, plus my own hand-checked vote), I'll list the domain in uc. I'll track the number of unique submissions in the TXT field. I'd suggest whitelisting through the usual channels. Suggestions (public or private) on the submission and inclusion criteria are most welcome. I've tried to capture the gist of what others have seemed to best agree on, but there is probably room for improvement. I can throw up a quick web site once things solidify a little. Once the list grows to sufficient size (maybe 100-200 domains?), we should be able to get a good idea of the usefulness. Once that happens, it will probably be more obvious how public we should make the data. - Ryan -- Ryan Thompson <ryan(a)sasknow.com> SaskNow Technologies - http://www.sasknow.com 901-1st Avenue North - Saskatoon, SK - S7K 1Y4 Tel: 306-664-3600 Fax: 306-244-7037 Saskatoon Toll-Free: 877-727-5669 (877-SASKNOW) North America

2 3

Re: [SURBL-Discuss] "Unconfirmed" list proof-of-concept
by Rob McEwen 03 Sep '04

03 Sep '04

Another good reason for the compromise I recommened in my last message is that me, Ryan, Chris, & others supporting unconfirmed.surbl.org would, essentially, get our list... it just wouldn't be servered automatically DNSRLB-style. But anyone would be free to copy this list and integrate it into their own private filtering rules as they desired. Someone might say... this is slower and less responsive than DNSRLB checking... I'd respond that it is the hardcore spammers that Jeff is referring to that are the fast moving target. FlowGo types tend to stay put with the same domains anyway (or, least they don't trade them out ever few hours or days). Also, wouldn't this list be higher quality with our combined efforts rather than just a few of us working on our own? Rob McEwen

1 0

Re: [SURBL-Discuss] "Unconfirmed" list proof-of-concept
by Rob McEwen 03 Sep '04

03 Sep '04

> If Ryan (or someone else) wants to take submissions directly, for > different lists, fine, but do not use the mailing list for this purpose, > the volume is high enough as it is now. > > Again, this mail is not trying to upset anyone, i think most of you will > understand our point of view, if not we would be willing to point that out > offlist. Even thought I vote FOR unconfirmed.surbl.org, I'd rather wait and let this issue work itself out than rush ahead and, as Jeff put it, "divide our efforts". Therefore, I have some constructive suggestions for how to unify our efforts in the meantime towards things I think we can all agree on which will only help achieve ALL of our goals, **even the ones that we disagree on :)** (I'll explain how...) FIRST: While we are waiting to see (for sure) what happens with unconfirmed.surbl.org, there are other ways to do greylisting. Please allow me to share some of the things that I do. (forgive me that some of these are unconventional and are not "by the book"... no lectures please). **I convert domains to IP addresses both in the body and headers and check these IP addresses and other IP address in the message against SpamHaus (they even suggest doing this in their literature... at least to some extent). **I check header URIs against SURBL (again, no lectures please... I do this more for auditing than for actual blocking) NOTE...I have actually found that these first two generate a high rate of spam with few false positives.... but there ARE enough false positives to keep me from using this for blocking in a "set it and forget it" mode, but... a high amount of spam "hits" nevertheless. **After my SpamHaus/DSBL MTA blocking is done, and after my linguistic filter & SURBL filtering is done... I then check the sending IP of each message still not blocked against both psbl.surriel.com and bl.spamcop.net . A lot of legitimate messages get "flagged" here... but still a rich auditing pool. This helps me quickly find much of what is getting past my filter without having to constantly dig through my client's e-mail. ...ALSO, in all these cases, I create bypass lists for each level of checking to prevent false "hits" on each of these checks so that I'm not constantly looking at the same stuff over and over again. I also blacklist obvious URIs in my linguistic filter that I deem not possibly ever legitimate. (I should probably be submitting these to SURBL? ... I have some follow questions on that process for another tread.) Looking at the data and what actually happens... my SpamHaus technique **IS** almost like what I think that people are looking for in unconfirmed.surbl.org SECOND: 5 days ago, I had a suggestion regarding integrating tracking of these Greymarketer organizations into the upcoming formal SURBL submission and tracking system we've talked about. The basic idea was to find a way to put pressure on them, generate an awareness of these groups, and try to help each other develop tools for blocking their spams without blocking the legitimate messages. Changing the subject slightly, another idea behind this was to work together to develop strategies for blocking spam from those proliferate spammers who often beat SURBL for a few hours or days with their new domains. For example, recall that "pesky porn spam" request I made last month... one of the guys on this list (forgot his name) gave me some suggestions and I develop a rule in my filter which catches ALL of these with zero chance of false positives. FINALLY: Overall, I suggest that there are plenty of things we can work on... even "out of the box" things... that we can all agree on. I suggest we work on these and shelve "unconfirmed.surbl.org" in the meantime. A good compromise is to take my suggestion and integrate tracking of things that would be good candidates for the UC list a a formal category in the format SURBL tracking system that we all agreed should be created. That way, if we later DO formally decide to do the UC list, the pro-UC folks will know that they are already further along in that task than we are now. Thoughts? BELOW IS MY EARLIER MESSAGE FROM ABOUT A WEEK AGO THAT I REFERRED TO: I think that there are two areas which present particular challenges. (1) e-mail marketers who play it "both ways"... thus making it hard to use SURBL to catch their bad behavior without blocking legitimate mail ...AND... (2) savy spammers who manage to get significant amounts through in the first few minutes/hours BEFORE getting blocked by SURBL... in particular, the ones who already use the best strategies to get around all other types of filtering. The quicker TTLs is helping with the savvy spammers. Also, I recall something about a newer version of SURBL which will use some kind of tracking to trace new domains back to older ones in order to attributing new spam to known and confirmed spammers so that they would stay "attached" to their previous bad records in order to blacklist them faster. What ever became of this? (Did I explain this correctly?) Anyway... even when these are done, we will STILL have some problems with the most savvy spammers. Also, I think that a lot of people fear that, as we work towards eliminating the rest of the FPs, more and more spam from these e-mail marketers who play it "both ways" will get through and the overall catch rate for SURBL may drop by 10 or 20 percent (or whatever). I'm willing to live with that... (gulp!) BUT... I think that it would be great to integrate into a formal tracking system a way to categorize URIs into either or both of these groups. ("SavySpams" and "GrayMarketer" ...or whatever) That way, we can use this data to help us form better "rules" in our linguistic/heuristic filters. The idea being that, at this point, the amount of spam that is getting through is much more focused than a large general pool of spam. This more narrow focus should give us the tools to close any loopholes that SURBL might not catch. I would also suggest that if a message's server address is already blocked by BOTH list.dsbl.org AND sbl-xbl.spamhaus.org, then it shouldn't be added to this particular list for the sake of keeping the list focused. It seems that, whatever the disagreements about RLBs are, I think that EVERYONE would agree with this particular standard as being a reliable (yet FP safe and conservative) standard for RBL blocking. I envision a "Gray page" which would list the top 10 offenders of Graymarketers who are bad enough to be mentioned, but not bad enough to get listed by SURBL and the top 10 "savvy spammers" who are known to periodically (abet temporarily) beat SURBL with their new domains. Subsequent offenders could be listed on following pages after the top 10 for each of these two categories. Each listing would include a link to more info about this spammer or series of spam. This more info page would also included samples of spam that hit real spamtraps (made anonymous), and, for the gray marketers, samples of legitimate mail with that particular URI. FP-safe rules would also be suggested... NOW... who would have the time to get all this together??? :) Rob McEwen

1 0

FP: autoscout24.xentranmailMUNGED.com
by Alex Broens 03 Sep '04

03 Sep '04

FP: autoscout24.xentranmailMUNGED.com * 5.0 OB_URI_RBL URI's domain appears in ws database at ob.surbl.org * [autoscout24.xentranmailMUNGED.com is blacklisted in] [URI RBL at multi.surbl.org] http://www.autoscout24.ch/ AutoScout24 AG Bernstrasse 41 3175 Flamatt E-Mail: info(a)autoscout24.ch Web: http://www.autoscout24.ch WHOIS: http://www.switch.ch/search/whois_form.html?Query=autoscout24.ch&Server=who… very white hat Pls whitelist & remove ASAP. thanks Alex

2 4

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Discuss