sc2.surbl.org, the improved version of the SpamCop SURBL list, is ready for testing. So is the new version of xs.surbl.org, which is now more accurate, has far fewer FPs, etc.
sc2 adds resolved IP checks, meaning sites hosted on the same networks are detected immediately upon the first report. It also means that folks should continue to use SpamCop reporting if they want to contribute to a very powerful SURBL list. Your SpamCop reports now have even more power in sc2. In cases of the worst spammers, SpamCop reporting leads to essentially immediate listing in sc2.
sc2 is on about 15 public nameservers and xs is on 22. That's probably not enough for running large production servers on, but it should be plenty for corpus checks and mail servers with small to medium message volumes.
If you have rsync access to the SURBL zone files you can also mirror the files locally for testing of course. The sc2 and xs zones are currently available via rsync. (If you have a large volume mail server, please apply for rsync access so that you can mirror the zone files locally: http://www3.surbl.org/rsync-signup.html and offload the public nameservers.)
After sc2 is tested for a while we will turn it into the production sc.surbl.org list, assuming it has better performance than the current list, which seems quite likely. At that point sc2 will go away, since it will have become sc.
xs may go into the 128th bit of multi.surbl.org if it tests well.
Please test sc2 and the revised xs and let us know how they perform for you. Those with large spam and ham corpora (such as the SpamAssassin developers) are encouraged to test and please let us know.
Here are SpamAssassin 3.0.1 and later configs for using these two lists:
urirhsbl URIBL_SC2_SURBL sc2.surbl.org. body URIBL_SC2_SURBL eval:check_uridnsbl('URIBL_SC2_SURBL') describe URIBL_SC2_SURBL Has URI in SC2 at http://www.surbl.org/lists.html tflags URIBL_SC2_SURBL net
score URIBL_SC2_SURBL 3.0
urirhsbl URIBL_XS_SURBL xs.surbl.org. body URIBL_XS_SURBL eval:check_uridnsbl('URIBL_XS_SURBL') describe URIBL_XS_SURBL Has URI in XS - Testing tflags URIBL_XS_SURBL net
score URIBL_XS_SURBL 2.0
SpamAssassin 2.64 rules and scores using SpamCopURI 0.22 or later look like this:
uri SC2_URI_RBL eval:check_spamcop_uri_rbl('sc2.surbl.org','127.0.0.2') describe SC2_URI_RBL Has URI in SC2 at http://www.surbl.org/lists.html tflags SC2_URI_RBL net
score SC2_URI_RBL 3.0
uri XS_URI_RBL eval:check_spamcop_uri_rbl('xs.surbl.org','127.0.0.2') describe XS_URI_RBL Has URI in XS - Testing tflags XS_URI_RBL net
score XS_URI_RBL 2.0
Jeff C. -- Don't harm innocent bystanders.
jdow pointed out problems with the prior rules for SA 3.0.1+. These ones should work:
urirhsbl URIBL_SC2_SURBL sc2.surbl.org. A 127.0.0.2 body URIBL_SC2_SURBL eval:check_uridnsbl('URIBL_SC2_SURBL') describe URIBL_SC2_SURBL Has URI in SC2 at http://www.surbl.org/lists.html tflags URIBL_SC2_SURBL net
score URIBL_SC2_SURBL 3.0
urirhsbl URIBL_XS_SURBL xs.surbl.org. A 127.0.0.2 body URIBL_XS_SURBL eval:check_uridnsbl('URIBL_XS_SURBL') describe URIBL_XS_SURBL Has URI in XS - Testing tflags URIBL_XS_SURBL net
score URIBL_XS_SURBL 2.0
Jeff C. -- Don't harm innocent bystanders.
OK the prior rules were still wrong. These will work:
urirhsbl URIBL_SC2_SURBL sc2.surbl.org. A body URIBL_SC2_SURBL eval:check_uridnsbl('URIBL_SC2_SURBL') describe URIBL_SC2_SURBL Has URI in SC2 at http://www.surbl.org/lists.html tflags URIBL_SC2_SURBL net
score URIBL_SC2_SURBL 3.0
urirhsbl URIBL_XS_SURBL xs.surbl.org. A body URIBL_XS_SURBL eval:check_uridnsbl('URIBL_XS_SURBL') describe URIBL_XS_SURBL Has URI in XS - Testing tflags URIBL_XS_SURBL net
score URIBL_XS_SURBL 2.0
Lints just fine on our SA3 with A and no addresses or numbers. (A is preferred over TXT.)
Note that we're using urirhsbl not urirhssub since sc2.surbl.org and xs.surbl.org are standalone lists (for testing) and not part of multi.surbl.org.
These lists will eventually go away as standalone lists, to very likely go into multi instead. Then you'll need to delete the sc2 rule and change xs to urirhssub and multi. We'll send an official announcement on the SURBL announcement list when this actually happens:
http://lists.surbl.org/mailman/listinfo/announce
Until then, please test sc2 and xs and let us know how they work for you.
Jeff C. -- Don't harm innocent bystanders.
Jeff Chan wrote:
Please test sc2 and the revised xs and let us know how they perform for you. Those with large spam and ham corpora (such as the SpamAssassin developers) are encouraged to test and please let us know.
I've had xs in my config for a while and with a low score (<1) it's hit on 28,362 emails that were tagged as spam and 11 not tagged over the last 2 weeks. The hits are pretty consistent over that time frame. I'm not sure when xs was "revised" so the numbers might not reflect current performance. Total mail volume during that time is ~100,000 with 75% tagged as spam.
Of the 11 not tagged, 3 were from mailing lists about spam, 2 were on a whitelisted mailing list (probably fps), 4 were fns, and 1 was somewhat questionable (probably an fp). Unfortunately, I don't have a way of seeing what domains hit so I can't know what the fps were.
Of the 28,373 messages that were hits: 28,176 also hit one of the RAZOR2 rules 28,049 also hit URIBL_BLACK 26,767 also hit URIBL_JP_SURBL 25,912 also hit URIBL_SBL and the remaining SURBL (AB,SC,OB,WS) hit between 22,000 and 25,000 messages.
Compared to other URIBL it ranks last in total spam hits (shown as tagged spam hits/not tagged spam hits): RAZOR2 68,015/810 (included for comparison even though it's not a URIBL, it hits the most spam test at my site) URIBL_BLACK 55,748/243 URIBL_SBL 54,721/167 URIBL_JP_SURBL 49,221/21 URIBL_AB_SURBL 36,673/3 URIBL_SC_SURBL 34,322/5 URIBL_OB_SURBL 32,970/75 URIBL_WS_SURBL 32,929/88 URIBL_XS_SURBL 28,366/11
The hit rate is a little low, but the approximate fp ratio ratio is very good. I'm a fan of anything that moves scores upwards!
Daniel
On Wednesday, July 27, 2005, 12:15:23 AM, Daniel Kleinsinger wrote:
I've had xs in my config for a while and with a low score (<1) it's hit on 28,362 emails that were tagged as spam and 11 not tagged over the last 2 weeks. The hits are pretty consistent over that time frame. I'm not sure when xs was "revised" so the numbers might not reflect current performance.
The list content change on XS was made on 7/25. The revised version of XS should hit more spam and much less ham.
Total mail volume during that time is ~100,000 with 75% tagged as spam.
Of the 11 not tagged, 3 were from mailing lists about spam, 2 were on a whitelisted mailing list (probably fps), 4 were fns, and 1 was somewhat questionable (probably an fp). Unfortunately, I don't have a way of seeing what domains hit so I can't know what the fps were.
Of the 28,373 messages that were hits: 28,176 also hit one of the RAZOR2 rules 28,049 also hit URIBL_BLACK 26,767 also hit URIBL_JP_SURBL 25,912 also hit URIBL_SBL and the remaining SURBL (AB,SC,OB,WS) hit between 22,000 and 25,000 messages.
Compared to other URIBL it ranks last in total spam hits (shown as tagged spam hits/not tagged spam hits): RAZOR2 68,015/810 (included for comparison even though it's not a URIBL, it hits the most spam test at my site) URIBL_BLACK 55,748/243 URIBL_SBL 54,721/167 URIBL_JP_SURBL 49,221/21 URIBL_AB_SURBL 36,673/3 URIBL_SC_SURBL 34,322/5 URIBL_OB_SURBL 32,970/75 URIBL_WS_SURBL 32,929/88 URIBL_XS_SURBL 28,366/11
The hit rate is a little low, but the approximate fp ratio ratio is very good. I'm a fan of anything that moves scores upwards!
Thanks much for sharing your results!
Has anyone else given XS or SC2 a try yet? It's ok to respond after some more data has accumulated if you're waiting for that.
Jeff C. -- Don't harm innocent bystanders.
Some stats from one of our SA servers. After about two days we had:
9076 SURBL hits 5373 SC2 hits 4813 SC hits 1148 SC2 hits that did not also hit SC 588 SC hits that did not also hit SC2 3701 XS hits 1890 SC2 hits that did not hit XS 218 XS hits that did not hit SC2
So it looks like sc2 hit about 10% more messages than SC.
Of the other lists:
7779 JP 6781 OB 5798 WS 4691 AB 7 PH
This is without analysis of FPs.
Would be very interested to hear how these new lists test out SpamAssassin corpora, or any other corpora or mail servers for that matter.
Jeff C. -- Don't harm innocent bystanders.
On Monday 25 July 2005 01:14 am, Jeff Chan wrote:
Please test sc2 and the revised xs and let us know how they> perform for you. Those with large spam and ham corpora (such as> the SpamAssassin developers) are encouraged to test and please> let us know.
Although I don't have a large amount of mail received at my home system, SC2 is scoring fairly well: TOP SPAM RULES FIRED------------------------------------------------------------RANK RULE NAME COUNT %OFRULES %OFMAIL %OFSPAM %OFHAM------------------------------------------------------------ 1 PYZOR_CHECK 130 5.87 71.04 100.00 100.00 2 DIGEST_MULTIPLE 119 5.38 65.03 91.54 0.00 3 RAZOR2_CF_RANGE_51_100 116 5.24 63.39 89.23 0.00 4 RAZOR2_CHECK 116 5.24 63.39 89.23 0.00 5 BAYES_99 112 5.06 61.20 86.15 0.00 6 URIBL_JP_SURBL 79 3.57 43.17 60.77 0.00 7 DCC_CHECK 73 3.30 39.89 56.15 0.00 8 URIBL_SC2_SURBL 71 3.21 38.80 54.62 0.00 9 URIBL_OB_SURBL 70 3.16 38.25 53.85 0.00 10 HTML_MESSAGE 66 2.98 36.07 50.77 7.55 11 URIBL_AB_SURBL 63 2.85 34.43 48.46 0.00 12 URIBL_SC_SURBL 56 2.53 30.60 43.08 0.00 13 URIBL_SBL 56 2.53 30.60 43.08 0.00 14 URIBL_XS_SURBL 56 2.53 30.60 43.08 0.00 15 RCVD_IN_XBL 55 2.49 30.05 42.31 0.00 16 RCVD_IN_BL_SPAMCOP_NET 50 2.26 27.32 38.46 0.00 17 URIBL_WS_SURBL 46 2.08 25.14 35.38 0.00 18 RCVD_NUMERIC_HELO 35 1.58 19.13 26.92 0.00 19 RCVD_IN_SORBS_DUL 30 1.36 16.39 23.08 0.00 20 DNS_FROM_RFC_POST 30 1.36 16.39 23.08 5.66 Chris -- ChrisRegistered Linux User 283774 http://counter.li.org06:15:07 up 5 days, 7:16, 1 user, load average: 0.18, 0.21, 0.26Mandriva Linux 10.1 Official, kernel 2.6.8.1-12mdk~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Pinking shears get dull just by looking at them -- Murphy's Laws of Sewing n°17~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
From the last three days:
SpamAssassinRuleHits for SPAM (score 10 and higher): BAYES_99 ( 95%) RAZOR2_CHECK ( 90%) RAZOR2_CF_RANGE_51_100 ( 85%) DIGEST_MULTIPLE ( 74%) URIBL_BLACK ( 72%) HTML_MESSAGE ( 71%) DCC_CHECK ( 66%) URIBL_OB_SURBL ( 60%) URIBL_JP_SURBL ( 60%) URIBL_WS_SURBL ( 57%) URIBL_SC2_SURBL ( 57%) <-- PYZOR_CHECK ( 55%) URIBL_SBL ( 52%) URIBL_SC_SURBL ( 50%) URIBL_XS_SURBL ( 44%) <-- URIBL_AB_SURBL ( 43%) MIME_HTML_ONLY ( 40%) RCVD_IN_SORBS_DUL ( 39%) FORGED_OUTLOOK_TAGS ( 31%) RCVD_IN_NJABL_DUL ( 30%)
Kind Regards, Sander Holthaus
From the last few days:
SURBL Hits --------------- ------- URIBL_PH_SURBL 3 URIBL_AB_SURBL 5,342 URIBL_XS_SURBL 3,529 URIBL_JP_SURBL 14,423 URIBL_SC2_SURBL 5,681 URIBL_OB_SURBL 11,742 URIBL_SC_SURBL 5,097 URIBL_WS_SURBL 9,931