Release 0.14 is up on sourceforge.net.
Sorry about all these releases. This is just a small
one to fix the 'Modification of non-creatable array value attempted'
error a few people have been seeing.
You can fetch it at http://sourceforge.net/projects/spamcopuri
--eric
I have just released SpamCopURI 0.13. This release adds some
basic query caching as well as registrar based lookups against
the RBL. Instead of querying for both the 2nd and 3rd level
domain we are now only querying for the 2nd OR the 3rd based
on the TLD. Let me know if you see any issues with this: false
positives, misses that should be hits, etc.
We are now caching the query results on a per test basis
so multiple URLs to the same to domain should only get checked
once per RBL.
There is also a change that fixes a compile error under
Perl 5.005 reported on this list.
Open redirect resolution is still off by default. I would
like to hear how this has worked out for those that have
enabled it. Do you think its safe enough to enabled it
by default?
The latest release can be retrieved from
http://sourceforge.net/projects/spamcopuri
--eric
BigEvil.cf and MidEvil.cf are now available in SURBL form as
be.surbl.org, for use with SpamCopURI SpamAssassin 2.63 and
URIDNSBL SpamAssassin 3.0 plugins. Thanks Chris, Paul and
Gary Funck!
Here's an excerpt about the new list from the Quick Start
section:
http://www.surbl.org/
Chris Santerre and Paul Barbeau's BigEvil and MidEvil
SpamAssassin rules are now available as an SURBL for use with
plugins and programs such as those mentioned above which can
extract message body URI domains and compare them against
name-based RBLs. The name of the list is be.surbl.org, and some
sample rules and scores to use it appears below. The well-known
and popular BigEvil and MidEvil SA rulesets are used to block
messages based on domains that have occurred in spam message body
URIs. Using this as an SURBL instead allows you to remove this
relatively large ruleset from SA memory and lets DNS cache the
data in a zone file instead, querying SURBL hits from DNS as
needed.
An SA 2.63 rule and score using SpamCopURI (but not the SpamCop
data!) looks like this:
uri BE_URI_RBL eval:check_spamcop_uri_rbl('be.surbl.org','127.0.0.2')
describe BE_URI_RBL URI's domain appears in BigEvil
tflags BE_URI_RBL net
score BE_URI_RBL 3.0
An SA 3.0 rule and score using URIBL's urirhsbl looks like this:
urirhsbl URIBL_BE_SURBL be.surbl.org. A
header URIBL_BE_SURBL eval:check_uridnsbl('URIBL_BE_SURBL')
describe URIBL_BE_SURBL Contains a URL listed BigEvil
tflags URIBL_BE_SURBL net
score URIBL_BE_SURBL 3.0
be.surbl.org can be used alone or with other SURBL lists; all
that's needed are different rule and score names, as we've shown
in the samples. More information about be.surbl.org can be found
in the Additional SURBLs section.
http://www.surbl.org/additional.htmlbe.surbl.org joins Bill Stearns' sa-blacklist-based ws.surbl.org
and my own SpamCop URI-based sc.surbl.org SURBLs. All are
described more at the site.
Please send me any questions, comments, corrections, updates,
etc.
Cheers,
Jeff C.
P.S. We will probably offer a combined list at some point.
We're still working out the details of that. Until then it's
quite possible to use one or more of the lists simply by using
separate SA rules for each one that you want to use, as shown
in the Quick Start samples.
P.P.S. The sample rules have been updated to mention "SpamCop"
only in the descriptions of rules that actually use SpamCop data.
--
Jeff Chan
mailto:jeffc@surbl.org
http://www.surbl.org/
Just release 0.12 to fix a test some users may have had errors with
during make test. No real need to grab this unless you want a clean make
test.
--eric
I have just released SpamCopURI version 0.11. This fixes a few
bugs that had been reported and adds open redirect resolution.
This basically takes a URL from say rd.yahoo.com and attempts
to resolve the Location header without ever fetching from
the potential spammy site.
Only the URLs that have hosts that match an address list get
redirect resolution. As well, redirect resolution is off
by default, but can be enabled in the conf file. I have
placed several open redirect sites in the conf file.
The basic requirement is that the redirect return a 300
level HTTP response when fetching. I placed google.com
in there even though they don't have their own redirect
domain, but this should be fairly safe since most if not
all google URLs are either redirects or searches. Give
it a try and tell me what you think. This is all dependent
upon LWP, but if you don't have LWP everything else
will function as it did before.
I have removed all the deprecated tests that depended on local Storable
data. See the INSTALL file for information about upgrading
from a previous version. There is also a bit more information
about installation that should help those that had trouble
in the past.
--eric
Here are a couple changes to the SURBL testpoints and TXT
records:
http://www.surbl.org/news.html
4/20/04: SURBL testpoints "example.com" have been changed to
"surbl-org-permanent-test-point.com" to avoid potential spam
detection on sample URIs.
So the SURBL testpoints are now:
Name: test.sc.surbl.org.sc.surbl.org (with ws for sc, etc.)
Address: 127.0.0.2
Name: test.surbl.org.sc.surbl.org
Address: 127.0.0.2
Name: 2.0.0.127.sc.surbl.org
Address: 127.0.0.2
Name: surbl-org-permanent-test-point.com.sc.surbl.org
Address: 127.0.0.2
4/20/04: TXT record for sc.surbl.org is changed from:
"Message body contains recently and multiply-reported SpamCop
spamvertised domain."
to:
"Message body contains SpamCop spamvertised domain."
TXT record for ws.surbl.org is changed from:
"Message body contains domain in sa-blacklist. See:
http://www.stearns.org/sa-blacklist/"
to:
"Blocked, See: http://www.stearns.org/sa-blacklist/".
Please update any test suites, rules, configs and other code
accordingly. We expect these to be (more) stable. :-)
(The default/sample SA 2.63 SpamCopURI and SA 3.0 urirhsbl rules
seem to write their own text descriptions based on the A record,
so these changes probably do not affect most SpamAssassin users
of SURBLs. Other applications may need to make adjustments if
they were using the TXT records.)
Jeff C.
--
Jeff Chan
mailto:jeffc@surbl.org
http://www.surbl.org/
We've changed the list settings slightly to direct replies to
this announcement list to go to the discussion list, and for
replies on the discussion list to go to the list by default
rather than to the poster.
discuss(a)lists.surbl.org
Hope that's ok with everyone. Please reply off list if it's
not. :-)
Jeff C.
--
Jeff Chan
mailto:jeffc@surbl.org
http://www.surbl.org/
We're made a document describing some of the general properties
which code using SURBLs should have in order to use the data as
it was designed and intended. We hope these comments may be
useful to developers. Our Implementation Guidelines are brief
and copied below.
http://www.surbl.org/implementation.html
Implementation Guidelines
Here are some very brief guidelines for folks writing software to
use SURBL lists. Your code should:
1. Extract URIs from message bodies. (Extraction of URIs from
message bodies should ideally include full resolution of
redirections into the final target domain name. This can be a
non-trivial problem.)
2. Extract base (registrar) domains from those URIs. This
includes removing any and all leading host names, subdomains,
www., randomized subdomains, etc. In order to determine the base
domain it may be necessary to use a table of country code TLDs
(ccTLDs) such as the partially-imcomplete one SURBL uses.
3. Not do name resolution on the domains.
4. Look up the domain name in the SURBL by prepending it to
the name of the SURBL, e.g., domainundertest.com.sc.surbl.org,
then doing Address record DNS resolution on the resulting
combined name. A non-result indicates lack of inclusion in the
list. A result of 127.0.0.2 represents inclusion, i.e., probable
spam.
5. Handle numeric IPs in URIs similarly, but reverse the octet
ordering before comparison against the RBL. This is standard
practice for RBLs. For example, http://1.2.3.4/ is checked as
4.3.2.1.sc.surbl.org.
SURBL lists unusually have both names and numbers in the same
list. For example, 2.0.0.127 and test.surbl.org and similar
actual spam domains and addresses are both in all SURBL lists.
Numbered addresses in SURBLs should have occurred in spams as
numbers, e.g.: literally http://1.2.3.4/. Additional SURBL test
points are mentioned in the News & Notes section.
__
Please send me any comments, updates, revisions, corrections,
questions, etc...
Jeff C.
--
Jeff Chan
mailto:jeffc@surbl.org
http://www.surbl.org/
I've udpated the SURBL web site to use frames and have freshened
the content slightly. Please let me know if you spot any broken
links, etc.
http://www.surbl.org/
Also added "An Open Letter To Operators Of Redirection Sites"
in which we try to appeal to redirection sites to deny their
services to spam URI domains (e.g., spammers' web sites).
Redirection sites may become an increasing problem if we're
successful in blocking spams with their sites directly linked.
http://www.surbl.org/redirect.html
Comments, revisions, questions, suggestions on any of that are
welcomed.
Cheers,
Jeff C.
--
Jeff Chan
mailto:jeffc@surbl.org
http://www.surbl.org/
I probably should have introduced this second SURBL list
that can be used together with or in place of sc.surbl.org
before mentioning that its name was changing from sa.surbl.org
to ws.surbl.org. :-) Note that the two lists have different
data sources, so strictly speaking one is not a replacement for
the other. They're two different lists. sc uses URI domains
from SpamCop reports. The data source for ws data is described
below. Both lists have merits and we'd encourage you to consider
trying both.
Here's an announcement with the additional update that
we've changed the *sample rule names* for the ws list to use
"WS" instead of "SA":
__
http://www.surbl.org/ (with some live links)
More SURBL lists
In addition to the first SpamCop URI-derived SURBL sc.surbl.org, we
are pleased to host another RBL compatible with the SpamCopURI or
URIDNSBL SpamAssassin plugins, or any other software that can
check message body domains against a name-based RBL. Data for the
second SURBL ws.surbl.org comes from the domains in Bill Stearns'
SpamAssassin blacklist: sa-blacklist. This is a large list of
spam domains, including those found in spam message body URIs.
Both ws.surbl.org and sc.surbl.org SURBLs can be used in the same
SA installation by using two sets of rules.
An SA 2.63 rule and score using SpamCopURI (but not the SpamCop
data!) looks like this:
uri WS_URI_RBL eval:check_spamcop_uri_rbl('ws.surbl.org','127.0.0.2')
describe WS_URI_RBL URI's domain appears in spamcop database at ws.surbl.org
tflags WS_URI_RBL net
score WS_URI_RBL 3.0
An SA 3.0 rule and score using URIBL's urirhsbl looks like this:
urirhsbl URIBL_WS_SURBL ws.surbl.org. A
header URIBL_WS_SURBL eval:check_uridnsbl('URIBL_WS_SURBL')
describe URIBL_WS_SURBL Contains a URL listed in the WS SURBL blocklist
tflags URIBL_WS_SURBL net
score URIBL_WS_SURBL 3.0
More details about ws.surbl.org are available in the section
"Additional SURBLs for spam URI testing" (copied below).
Please note that the name of this list is being changed from
sa.surbl.org to ws.surbl.org. If you were using the old name in
your rules please update them to the new name.
...
Additional SURBLs for spam URI testing
Additional SURBLs that list domains occurring in spam message
bodies may be used with the same routines that use the
sc.surbl.org RBL.
sa-blacklist available as RBL: ws.surbl.org
In cooperation with Bill Stearns, SURBL is making his
sa-blacklist SpamAssassin blacklist available as the RBL
ws.surbl.org. It can be used in the same way as sc.surbl.org, for
example by adding urirhsbl and SpamCopURI rules as described in
the Quick Start section at the top of this document. Like sc,
ws.surbl.org is available through DNS and, for large-volume mail
servers, as rsynced BIND and rbldns zone files. Raymond
Dijkxhoorn has graciously agreed to host the ws.surbl.org zone
files from his rsync server along with sc.surbl.org's. Please
contact him at rsync(a)surbl.org for rsync access.
Both sc and ws RBLs can be used in the same installation. The
choice of using either or both or none is yours. Their data
differs somewhat, and we'll try to briefly describe and link some
of the differences here. Bill's list is rather large at about
9600 domains. It consists of domains found in spam message body
URIs and some spam sender and spam operator domains. Given that
the former are more relevant to isolate these days, most of the
recent additions to Bill's list have been URI domains. Those are
also the domains most relevant for use with the message body
checking approach which we propose throughout this site.
The data in sa-blacklist and therefore ws.surbl.org differ from
the SpamCop URI report data described above in that the list is
about ten times larger, more stable, and may have a slightly
higher false positive rate. Bill's policy for inclusion and
cleaning of the sa-blacklist is quite sound, however, so folks
should feel comfortable giving this list a try in addition to the
sc list. ws may currently detect some spam that sc misses, and
vice versa, but it's worth mentioning that the current sc is a
working prototype and that we expect the performance of sc to
improve as we tune the sc data engine further. sc just got out of
the gate, yet it already has some worthy competition in ws.
Thanks Bill!
Because ws is larger and more stable, the zone files for it gets
a six hour TTL compared to 10 minutes for sc. Due to the
differences between the time scales, sizes, and data sources of
ws and sc, we probably won't be offering a combined ws plus sc
list. For example it would be difficult to say what TTL a merged
list should get, and you probably would not want a megabyte plus
BIND zone file refreshing every 10 minutes. For those using
rsynced zone files that would probably not be an issue, but for
those using BIND, the DNS traffic quite well could be.
We encourage you to give ws.surbl.org a try.
Please note that the name of this list is being changed from
sa.surbl.org to ws.surbl.org. If you were using the old name in
your rules please update them to the new name.
Jeff C.
--
Jeff Chan
mailto:jeffc@surbl.org
http://www.surbl.org/