On Tuesday, June 15, 2004, 8:51:40 AM, Smart,Dan Smart,Dan wrote:
> What ever happened to the Anti-phishing rules on surbl. Never saw any
> domain come forward on this.
Hi Dan,
The anti-phishing data is in the combined list multi.surbl.org.
Because this is bitmasked data, as described on our lists page:
http://www.surbl.org/lists.html#multi
... some code needs to be written to decode the results. Hopefully
the SA coders will perhaps get a chance to get support for this
into the 3.0.0 release.
Jeff C.
--
Jeff Chan
mailto:jeffc@surbl.org
http://www.surbl.org/
Here's a test on my last 4 days of ham and spam. The T_DNS_FROM* rules
are using the envelope sender (that is, the MAIL FROM added by my MTA to
the message headers). They have a reasonably good hit rate (6.43% of
spam hit one of tested SURBL zones) and a 0% FP rate in this test. Only
7 out of 103 of those did not hit one of the URIBL rules, but they did
do it with zero FPs (I get FPs mostly because people discuss spam
domains in some of the ham tested here. That's another issue, though.)
Maybe it would be worthwhile factoring this into future development.
That is, also list known spammer envelope senders domains, maybe get
SpamCop to provide lists for that too, I suspect there's some overlap in
the other direction as well.
OVERALL% SPAM% HAM% S/O RANK SCORE NAME
2598 1602 996 0.617 0.00 0.00 (all messages)
100.000 61.6628 38.3372 0.617 0.00 0.00 (all messages as %)
3.580 5.8052 0.0000 1.000 1.00 0.01 T_DNS_FROM_SURBL_WS
0.885 1.4357 0.0000 1.000 0.67 0.01 T_DNS_FROM_SURBL_SC
9.161 14.5443 0.5020 0.967 0.56 1.00 URIBL_BE_SURBL
36.066 57.5531 1.5060 0.974 0.44 1.00 URIBL_WS_SURBL
0.423 0.6866 0.0000 1.000 0.33 0.01 T_DNS_FROM_SURBL_BE
29.908 47.5655 1.5060 0.969 0.11 1.00 URIBL_SC_SURBL
How's the multi roll-out going? It would definitely be handy for this
test (the code to support it already exists).
Daniel
--
Daniel Quinlan
http://www.pathname.com/~quinlan/
I've seen tons of email on this list about surbl and uribl, but I haven't
really made sense of it yet.
Is ther a FAQ on this stuff for SA users? Does it deal with performance
issues?
thanks,
-chuck
>-----Original Message-----
>From: Jeff Chan [mailto:jeffc@surbl.org]
>Sent: Monday, June 14, 2004 7:13 PM
>To: SpamAssassin Developers; SURBL Discuss
>Subject: [SURBL-Discuss] Re: Which rules are replaces by *.surbl.org?
>
>
>On Monday, June 14, 2004, 2:03:11 PM, Chris Santerre wrote:
>>>From: Jeff Chan [mailto:jeffc@surbl.org]
>
>>>Ideally if folks want every function, they should:
>>>
>>>0. Use sc.surbl.org
>>>
>>>1. Use ws.surbl.org (which now has the be.surbl.org domains)
>>>
>>>2. *Not* use be.surbl.org (which is now redundant)
>>>
>>>3. Use BigEvil.cf (and perhaps MidEvil.cf also, depending
>>>on how Chris and Paul work things out.)
>
>> Yes, but I want to add that there _WILL_ be a "BigEvil
>style" cf version of
>> ws.surbl.org for those people who won't/can't use the SURBL
>net lookups for
>> some strange reason. This is still being worked on. One of
>the main reasons
>> I haven't updates BE in a while is because I've been working
>on the new WS
>> submission stuff. (Thanks to everyone who is involved in that!)
>
>Thanks, I forgot about that other direction of rules style
>entries moving from sa-blacklist back into bigevil.cf. Sounds
>like the best of both worlds in a nice mirror of both types:
>
>A. ws.surbl.org gets all the "static" domains from BigEvil,
>sa-blacklist, etc. in the form of a SURBL.
>
>B. BigEvil.cf gets all the domains, including those from
>sa-blacklist, heavily wildcarded ones, etc. in the form of a
>ruleset.
>
>I didn't think of it earlier, but that will increase the
>overlapped coverage for folks using both of the above however.
>
>Please be sure to let me know when you start feeding the larger
>lists into BigEvil.cf so I can know when to stop feeding them
>into be. Don't want a feedback loop of those going into ws.
>Accordingly I will also stop feeding be into ws at that time.
>
>(Bill Stearns, please note the above. My feed of be to you
>should freeze at some point. Chris we should definitely
>coordinate when I should freeze the be I send Bill. Please
>let me know.)
Well I haven't even been updating BE. My update today removed some FP and
regex goofs. Nothing added. I have been adding to [ws] because it is MUCH
easier :D You can pretty much stop feeding [be] into [ws] right now.
>
>Also: *when should we announce that be domains are now in ws, and
>that people should stop using be?* Is everyone comfortable that
>the combined ws is now working as expected, including the be
>domains being folded in?
You could do that now. Stating that [be] will not change until the [ws] to
bigevil.cf script work is complete. Once that is done I will add the dynamic
stuff to BE. But from now on, I only update static domains to [ws].
>> Things should also pickup in the addition of new domains.
>More _trusted_
>> sources are being worked on now. We are being really picky,
>and making
>> people walk the Gauntlet of Fire! :D
>
>More data sources sound good. I'm glad we're being very careful
>that false positives don't get in. When we get a clearinghouse
>set up to double check them, that will help.
*cough* Paul you still alive? *cough*
--Chris
>-----Original Message-----
>From: Jeff Chan [mailto:jeffc@surbl.org]
>Sent: Sunday, June 13, 2004 12:28 AM
>To: SpamAssassin Users; SURBL Discuss
>Subject: Re: Which rules are replaces by *.surbl.org?
>
>
>On Saturday, June 12, 2004, 7:56:15 PM, ian list) wrote:
>> On Sat, 12 Jun 2004, Jeff Chan wrote:
>
>>> In other words the domains from BigEvil and MidEvil that can be
>>> listed without many wildcards go into ws.surbl.org and the
>>> domains that need more wildcards (too many to be practically
>>> enumerated) will end up in BigEvil.cf .
>>>
>>> Chris may not be ready to do the latter yet, but the former is
>>> already in place as of a few days ago. We're watching it all
>>> run for a while before announcing officially.
>
>> Thanks for info Jeff, one question, for us who rsync the
>zones off your
>> servers, will the be.surbl.org.bind/be.surbl.org.rbldsnd
>files disappear ?
>> Does this mean we need to reconfigure our bind/rbldnsd if BE
>disappears
>> and transforms back into a .cf ?
>
>We had lots of difficulty reaching people and getting them
>to stop using sa.surbl.org when we simply wanted to rename that
>list to ws.surbl.org, and that was early on, so I suspect
>be.surbl.org may live on but with essentially no content. be's
>been around longer so it would be harder to get it out of configs
>out there. But the useable content from be is now in ws.
>
>Ideally if folks want every function, they should:
>
>0. Use sc.surbl.org
>
>1. Use ws.surbl.org (which now has the be.surbl.org domains)
>
>2. *Not* use be.surbl.org (which is now redundant)
>
>3. Use BigEvil.cf (and perhaps MidEvil.cf also, depending
>on how Chris and Paul work things out.)
Yes, but I want to add that there _WILL_ be a "BigEvil style" cf version of
ws.surbl.org for those people who won't/can't use the SURBL net lookups for
some strange reason. This is still being worked on. One of the main reasons
I haven't updates BE in a while is because I've been working on the new WS
submission stuff. (Thanks to everyone who is involved in that!)
Things should also pickup in the addition of new domains. More _trusted_
sources are being worked on now. We are being really picky, and making
people walk the Gauntlet of Fire! :D
>
>That said, not every one chooses to use every component.
>The choice is up to them.
>
>To summarize the changes, the relatively fixed domains from
>be are now in ws, and the heavily wildcarded domains will
>end up only in BigEvil.cf. So to get the original BigEvil
>functionality one would continue to use BigEvil.cf and add
>ws.surbl.org. (That would also add the sa-blacklist domains
>for someone who was only using BigEvil before.)
Yeah I will be working on the more dynamic Bigevil.cf soon. I'm trying to
work on another ruleset now that has nothing to do with URLs :) My eyes
have been getting cross-eyed!
Chris Santerre
System Admin and SARE Ninja
http://www.rulesemporium.com
'It is not the strongest of the species that survives,
not the most intelligent, but the one most responsive to change.'
Charles Darwin
On Saturday, June 12, 2004, 4:10:30 PM, Matt Kettler wrote:
> At 08:26 PM 6/12/04 +0200, Matthias Keller wrote:
>>I just upgraded to 2.63 and installed the patch to use surbls
>>But I'm now not quite sure which .cf-rules I may remove now....?
>>sc.surbl.org -> replaces spamcop_top200.cf ?
> No.. spamcop_top200 is NOT URI based, it's Received: headers..
> The normal spamcop DNSBL (RCVD_IN_BL_SPAMCOP_NET) overlaps with
> spamcop_top200.cf not SURBL.
>>ws.surbl.org -> replaces blacklist-uri.cf, right?
> Yes.
>> -- but also blacklist.cf ?
> No. surbl ONLY does uri's.. WS's blacklist.cf is a sender-domain blacklist.
>>be.surbl.org -> that one I'm sure, it replaces bigevil (and midevil) .....
> Somewhat, although be.surbl.org is going away and even right now it doesn't
> (and cannot) contain all of bigevil.
Thanks for a good response Matt! You hit all the points
excellently.
Key is that SURBLs contain message body URI domains. This is
a very different approach from most RBLs which as you note list
sender domains or sending IP addresses. SURBLs don't go after
the sources of the messages, they go after the URIs in the
message bodies.
More information about the lists can be found at:
http://www.surbl.org/lists.html
> Eventualy JC, WS, And CS are going to get together and merge all the
> static-text stuff in bigevil over to WS's stuff, and bigevil will focus
> only on wide-range regex stuff, at which point it can't be surbl hosted and
> must be a .cf file. (DNS can't do regexes, just exact text match)
The enumerable domains in be.surbl.org are now being merged into
ws.surbl.org, so be.surbl.org is just about ready to go away,
with the heavily-wildcarded, widely-varying domains ending up
exclusively in BigEvil.cf.
In other words the domains from BigEvil and MidEvil that can be
listed without many wildcards go into ws.surbl.org and the
domains that need more wildcards (too many to be practically
enumerated) will end up in BigEvil.cf .
Chris may not be ready to do the latter yet, but the former is
already in place as of a few days ago. We're watching it all
run for a while before announcing officially.
Jeff C.
--
Jeff Chan
mailto:jeffc@surbl.org
http://www.surbl.org/
Here's another open redirection site to handle:
http://www.clubiran.com/url/click.php?url=http://www.loan-america.com/index…
I've whitelisted it in SURBLs. Anything special that happens in
URI parsing code or the programs that call it may want to take
this one into account.
I've also written the owners of the redirection site.
Jeff C.
In the latest rsync of the dns zones i got the file ob.surbl.org.bind as
well. it's a biggie, at 3884780 bytes. Anyone know the source of this
data?
- Ian
From: "Menno van Bennekom" > Hi,
>
> I get spam with a different URL, the redirect has only one '/':
> <a
>
href="http://rd.yahoo.com/oashoscy/*http:/hjktccbz.woodwheel.info/mn/num17">
>
>
> This is not recognised by BIZ_TLD (in this example my copy, INFO_TLD).
> I can change that in the regular expression.
> But I don't think SPAMCOP_URI_RBL recognizes it too because woodwheel is
> in the database but SA gives no hit.
> If you click on the link above it works, so it seems the one slash is
> possible.
> Can anyone confirm that one slash is not recognized?
This should work with SpamCopURI.
What version are you using?
Have you got entry like
spamcop_uri_resolve_open_redirects 1
open_redirect_list_spamcop_uri rd.yahoo.com *.rd.yahoo.com
in spamcop_uri.cf?
John
I've filed bug 3467 in SA's bugzilla
http://bugzilla.spamassassin.org/show_bug.cgi?id=3467
suggesting that uri_to_domain discount URI's which don't end in valid
TLD's. There are test cases in which SA's get_uri_list can pick up URI
of the form http://random.gif/ which will return random.gif as the
domain and get fed into the pool of candidate domains to check for.
I don't know that SpamCopURI's behaviour is with the testcases I've
filed
Regards, Yusuf