Announce

announce@lists.surbl.org

1 participants
191 discussions

[SURBL-Announce] ANNOUNCE: BigEvil.cf and MidEvil.cf are now available in SURBL form as be.surbl.org
by Jeff Chan 22 Apr '04

22 Apr '04

BigEvil.cf and MidEvil.cf are now available in SURBL form as be.surbl.org, for use with SpamCopURI SpamAssassin 2.63 and URIDNSBL SpamAssassin 3.0 plugins. Thanks Chris, Paul and Gary Funck! Here's an excerpt about the new list from the Quick Start section: http://www.surbl.org/ Chris Santerre and Paul Barbeau's BigEvil and MidEvil SpamAssassin rules are now available as an SURBL for use with plugins and programs such as those mentioned above which can extract message body URI domains and compare them against name-based RBLs. The name of the list is be.surbl.org, and some sample rules and scores to use it appears below. The well-known and popular BigEvil and MidEvil SA rulesets are used to block messages based on domains that have occurred in spam message body URIs. Using this as an SURBL instead allows you to remove this relatively large ruleset from SA memory and lets DNS cache the data in a zone file instead, querying SURBL hits from DNS as needed. An SA 2.63 rule and score using SpamCopURI (but not the SpamCop data!) looks like this: uri BE_URI_RBL eval:check_spamcop_uri_rbl('be.surbl.org','127.0.0.2') describe BE_URI_RBL URI's domain appears in BigEvil tflags BE_URI_RBL net score BE_URI_RBL 3.0 An SA 3.0 rule and score using URIBL's urirhsbl looks like this: urirhsbl URIBL_BE_SURBL be.surbl.org. A header URIBL_BE_SURBL eval:check_uridnsbl('URIBL_BE_SURBL') describe URIBL_BE_SURBL Contains a URL listed BigEvil tflags URIBL_BE_SURBL net score URIBL_BE_SURBL 3.0 be.surbl.org can be used alone or with other SURBL lists; all that's needed are different rule and score names, as we've shown in the samples. More information about be.surbl.org can be found in the Additional SURBLs section. http://www.surbl.org/additional.html be.surbl.org joins Bill Stearns' sa-blacklist-based ws.surbl.org and my own SpamCop URI-based sc.surbl.org SURBLs. All are described more at the site. Please send me any questions, comments, corrections, updates, etc. Cheers, Jeff C. P.S. We will probably offer a combined list at some point. We're still working out the details of that. Until then it's quite possible to use one or more of the lists simply by using separate SA rules for each one that you want to use, as shown in the Quick Start samples. P.P.S. The sample rules have been updated to mention "SpamCop" only in the descriptions of rules that actually use SpamCop data. -- Jeff Chan mailto:jeffc@surbl.org http://www.surbl.org/

1 0

ANNOUNCE: Mail::SpamAsssassin::SpamCopURI 0.12
by Eric Kolve 21 Apr '04

21 Apr '04

Just release 0.12 to fix a test some users may have had errors with during make test. No real need to grab this unless you want a clean make test. --eric

1 0

ANNOUNCE: Mail::SpamAsssassin::SpamCopURI 0.11
by Eric Kolve 21 Apr '04

21 Apr '04

I have just released SpamCopURI version 0.11. This fixes a few bugs that had been reported and adds open redirect resolution. This basically takes a URL from say rd.yahoo.com and attempts to resolve the Location header without ever fetching from the potential spammy site. Only the URLs that have hosts that match an address list get redirect resolution. As well, redirect resolution is off by default, but can be enabled in the conf file. I have placed several open redirect sites in the conf file. The basic requirement is that the redirect return a 300 level HTTP response when fetching. I placed google.com in there even though they don't have their own redirect domain, but this should be fairly safe since most if not all google URLs are either redirects or searches. Give it a try and tell me what you think. This is all dependent upon LWP, but if you don't have LWP everything else will function as it did before. I have removed all the deprecated tests that depended on local Storable data. See the INSTALL file for information about upgrading from a previous version. There is also a bit more information about installation that should help those that had trouble in the past. --eric

1 0

SURBL testpoint changed, TXT records shortened
by Jeff Chan 21 Apr '04

21 Apr '04

Here are a couple changes to the SURBL testpoints and TXT records: http://www.surbl.org/news.html 4/20/04: SURBL testpoints "example.com" have been changed to "surbl-org-permanent-test-point.com" to avoid potential spam detection on sample URIs. So the SURBL testpoints are now: Name: test.sc.surbl.org.sc.surbl.org (with ws for sc, etc.) Address: 127.0.0.2 Name: test.surbl.org.sc.surbl.org Address: 127.0.0.2 Name: 2.0.0.127.sc.surbl.org Address: 127.0.0.2 Name: surbl-org-permanent-test-point.com.sc.surbl.org Address: 127.0.0.2 4/20/04: TXT record for sc.surbl.org is changed from: "Message body contains recently and multiply-reported SpamCop spamvertised domain." to: "Message body contains SpamCop spamvertised domain." TXT record for ws.surbl.org is changed from: "Message body contains domain in sa-blacklist. See: http://www.stearns.org/sa-blacklist/" to: "Blocked, See: http://www.stearns.org/sa-blacklist/". Please update any test suites, rules, configs and other code accordingly. We expect these to be (more) stable. :-) (The default/sample SA 2.63 SpamCopURI and SA 3.0 urirhsbl rules seem to write their own text descriptions based on the A record, so these changes probably do not affect most SpamAssassin users of SURBLs. Other applications may need to make adjustments if they were using the TXT records.) Jeff C. -- Jeff Chan mailto:jeffc@surbl.org http://www.surbl.org/

1 0

[SURBL-Announce] Administrivia: announce replies go to discuss, discuss replies go to list
by Jeff Chan 21 Apr '04

21 Apr '04

We've changed the list settings slightly to direct replies to this announcement list to go to the discussion list, and for replies on the discussion list to go to the list by default rather than to the poster. discuss(a)lists.surbl.org Hope that's ok with everyone. Please reply off list if it's not. :-) Jeff C. -- Jeff Chan mailto:jeffc@surbl.org http://www.surbl.org/

1 0

SURBL Implemetation Guidelines
by Jeff Chan 20 Apr '04

20 Apr '04

We're made a document describing some of the general properties which code using SURBLs should have in order to use the data as it was designed and intended. We hope these comments may be useful to developers. Our Implementation Guidelines are brief and copied below. http://www.surbl.org/implementation.html Implementation Guidelines Here are some very brief guidelines for folks writing software to use SURBL lists. Your code should: 1. Extract URIs from message bodies. (Extraction of URIs from message bodies should ideally include full resolution of redirections into the final target domain name. This can be a non-trivial problem.) 2. Extract base (registrar) domains from those URIs. This includes removing any and all leading host names, subdomains, www., randomized subdomains, etc. In order to determine the base domain it may be necessary to use a table of country code TLDs (ccTLDs) such as the partially-imcomplete one SURBL uses. 3. Not do name resolution on the domains. 4. Look up the domain name in the SURBL by prepending it to the name of the SURBL, e.g., domainundertest.com.sc.surbl.org, then doing Address record DNS resolution on the resulting combined name. A non-result indicates lack of inclusion in the list. A result of 127.0.0.2 represents inclusion, i.e., probable spam. 5. Handle numeric IPs in URIs similarly, but reverse the octet ordering before comparison against the RBL. This is standard practice for RBLs. For example, http://1.2.3.4/ is checked as 4.3.2.1.sc.surbl.org. SURBL lists unusually have both names and numbers in the same list. For example, 2.0.0.127 and test.surbl.org and similar actual spam domains and addresses are both in all SURBL lists. Numbered addresses in SURBLs should have occurred in spams as numbers, e.g.: literally http://1.2.3.4/. Additional SURBL test points are mentioned in the News & Notes section. __ Please send me any comments, updates, revisions, corrections, questions, etc... Jeff C. -- Jeff Chan mailto:jeffc@surbl.org http://www.surbl.org/

1 0

[SURBL-Announce] SURBL web site updated, Letter to Redirectors added
by Jeff Chan 19 Apr '04

19 Apr '04

I've udpated the SURBL web site to use frames and have freshened the content slightly. Please let me know if you spot any broken links, etc. http://www.surbl.org/ Also added "An Open Letter To Operators Of Redirection Sites" in which we try to appeal to redirection sites to deny their services to spam URI domains (e.g., spammers' web sites). Redirection sites may become an increasing problem if we're successful in blocking spams with their sites directly linked. http://www.surbl.org/redirect.html Comments, revisions, questions, suggestions on any of that are welcomed. Cheers, Jeff C. -- Jeff Chan mailto:jeffc@surbl.org http://www.surbl.org/

1 0

[SURBL-Announce] Bill Stearns' sa-blacklist available as SURBL: ws.surbl.org
by Jeff Chan 17 Apr '04

17 Apr '04

I probably should have introduced this second SURBL list that can be used together with or in place of sc.surbl.org before mentioning that its name was changing from sa.surbl.org to ws.surbl.org. :-) Note that the two lists have different data sources, so strictly speaking one is not a replacement for the other. They're two different lists. sc uses URI domains from SpamCop reports. The data source for ws data is described below. Both lists have merits and we'd encourage you to consider trying both. Here's an announcement with the additional update that we've changed the *sample rule names* for the ws list to use "WS" instead of "SA": __ http://www.surbl.org/ (with some live links) More SURBL lists In addition to the first SpamCop URI-derived SURBL sc.surbl.org, we are pleased to host another RBL compatible with the SpamCopURI or URIDNSBL SpamAssassin plugins, or any other software that can check message body domains against a name-based RBL. Data for the second SURBL ws.surbl.org comes from the domains in Bill Stearns' SpamAssassin blacklist: sa-blacklist. This is a large list of spam domains, including those found in spam message body URIs. Both ws.surbl.org and sc.surbl.org SURBLs can be used in the same SA installation by using two sets of rules. An SA 2.63 rule and score using SpamCopURI (but not the SpamCop data!) looks like this: uri WS_URI_RBL eval:check_spamcop_uri_rbl('ws.surbl.org','127.0.0.2') describe WS_URI_RBL URI's domain appears in spamcop database at ws.surbl.org tflags WS_URI_RBL net score WS_URI_RBL 3.0 An SA 3.0 rule and score using URIBL's urirhsbl looks like this: urirhsbl URIBL_WS_SURBL ws.surbl.org. A header URIBL_WS_SURBL eval:check_uridnsbl('URIBL_WS_SURBL') describe URIBL_WS_SURBL Contains a URL listed in the WS SURBL blocklist tflags URIBL_WS_SURBL net score URIBL_WS_SURBL 3.0 More details about ws.surbl.org are available in the section "Additional SURBLs for spam URI testing" (copied below). Please note that the name of this list is being changed from sa.surbl.org to ws.surbl.org. If you were using the old name in your rules please update them to the new name. ... Additional SURBLs for spam URI testing Additional SURBLs that list domains occurring in spam message bodies may be used with the same routines that use the sc.surbl.org RBL. sa-blacklist available as RBL: ws.surbl.org In cooperation with Bill Stearns, SURBL is making his sa-blacklist SpamAssassin blacklist available as the RBL ws.surbl.org. It can be used in the same way as sc.surbl.org, for example by adding urirhsbl and SpamCopURI rules as described in the Quick Start section at the top of this document. Like sc, ws.surbl.org is available through DNS and, for large-volume mail servers, as rsynced BIND and rbldns zone files. Raymond Dijkxhoorn has graciously agreed to host the ws.surbl.org zone files from his rsync server along with sc.surbl.org's. Please contact him at rsync(a)surbl.org for rsync access. Both sc and ws RBLs can be used in the same installation. The choice of using either or both or none is yours. Their data differs somewhat, and we'll try to briefly describe and link some of the differences here. Bill's list is rather large at about 9600 domains. It consists of domains found in spam message body URIs and some spam sender and spam operator domains. Given that the former are more relevant to isolate these days, most of the recent additions to Bill's list have been URI domains. Those are also the domains most relevant for use with the message body checking approach which we propose throughout this site. The data in sa-blacklist and therefore ws.surbl.org differ from the SpamCop URI report data described above in that the list is about ten times larger, more stable, and may have a slightly higher false positive rate. Bill's policy for inclusion and cleaning of the sa-blacklist is quite sound, however, so folks should feel comfortable giving this list a try in addition to the sc list. ws may currently detect some spam that sc misses, and vice versa, but it's worth mentioning that the current sc is a working prototype and that we expect the performance of sc to improve as we tune the sc data engine further. sc just got out of the gate, yet it already has some worthy competition in ws. Thanks Bill! Because ws is larger and more stable, the zone files for it gets a six hour TTL compared to 10 minutes for sc. Due to the differences between the time scales, sizes, and data sources of ws and sc, we probably won't be offering a combined ws plus sc list. For example it would be difficult to say what TTL a merged list should get, and you probably would not want a megabyte plus BIND zone file refreshing every 10 minutes. For those using rsynced zone files that would probably not be an issue, but for those using BIND, the DNS traffic quite well could be. We encourage you to give ws.surbl.org a try. Please note that the name of this list is being changed from sa.surbl.org to ws.surbl.org. If you were using the old name in your rules please update them to the new name. Jeff C. -- Jeff Chan mailto:jeffc@surbl.org http://www.surbl.org/

1 0

Whitelisting log added
by Jeff Chan 15 Apr '04

15 Apr '04

Given the probable need to improve whitelisting, I've added a log of domains that would go onto sc.surbl.org but are then prevented from getting onto the list by the whitelist(s): http://www.surbl.org/whitelist-hits.new.log That goes along with the log of new additions to sc.surbl.org, i.e., essentially a blacklisting log: http://www.surbl.org/top-sites-domains.new.log I've also grabbed copy of 500 popular web site domains for addition to the whitelist. A couple of the recent whitelist hits have been from it. So far they seem reasonable. Whitelisting will continue in the next version of the engine, hopefully with some larger data sets. Blacklisting based on SpamCop URI domain data will hopefully be more stable and broader in the next version also. In other words, there should be significantly less activity on the blacklist log since the list itself will be more stable. (For example under the current system you may see some domains that come off the list then get back on it.... Pay no attention to the man behind the curtain... :-) There should be a lot less of that.) Jeff C. -- Jeff Chan mailto:jeffc@surbl.org http://www.surbl.org/

1 1

[SURBL-Announce] Like sc.surbl.org, ws.surbl.org works with any SURBL-compatible programs
by Jeff Chan 13 Apr '04

13 Apr '04

A question about whether SpamCopURI would support using the alternative SURBL ws.surbl.org came up, so I thought I'd address that for everyone. Any program that knows how to extract URIs from message bodies, then domains from the URIs, then compare those domains against an RBL can use any or all of the SURBL lists. Therefore SpamCopURI will work with ws.surbl.org just fine. (Noting of course that the ws results won't necessarily be related to the SpamCop-derived data in the sc list.) All you need to do is add a rule with the name of that list: uri SA_URI_RBL eval:check_spamcop_uri_rbl('ws.surbl.org','127.0.0.2') describe SA_URI_RBL URI's domain appears in spamcop database at ws.surbl.org tflags SA_URI_RBL net score SA_URI_RBL 3.0 (Likewise in SpamAssassin 3.0 with urirhsbl:) urirhsbl URIBL_SA_SURBL ws.surbl.org. A header URIBL_SA_SURBL eval:check_uridnsbl('URIBL_SA_SURBL') describe URIBL_SA_SURBL Contains a URL listed in the SA SURBL blocklist tflags URIBL_SA_SURBL net score URIBL_SA_SURBL 3.0 You can run either SURBL or both if you like. Note that ws has a higher spam detection rate (currently) but also a somewhat higher false positive rate than sc. Here's a corpus check Dan Quinlan ran: OVERALL% SPAM% HAM% S/O RANK SCORE NAME 11189 1200 9989 0.107 0.00 0.00 (all messages) 100.000 10.7248 89.2752 0.107 0.00 0.00 (all messages as %) 6.095 56.2500 0.0701 0.999 1.00 1.00 URIBL_SC_SURBL 6.855 59.7500 0.5006 0.992 0.98 1.00 URIBL_SBL 9.545 72.8333 1.9421 0.974 0.95 0.01 T_URIBL_SA_SURBL 0.116 0.5000 0.0701 0.877 0.58 0.01 T_URIBL_DSBL SA_SURBL above reflects the old name for ws; SC_SURBL is sc.surbl.org. ws detected ~73% of spams in the spam corpus with a ~1.9% FP rate in the ham corups. sc detected ~56% with a <0.1% FP rate. We're still tuning how the SpamCop data is used, so the sc hit rates should improve and FPs decrease hopefully in the next version of the sc data engine. Cheers, Jeff C. -- Jeff Chan mailto:jeffc@surbl.org http://www.jeffchan.com/

1 0

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Announce