Discuss

discuss@lists.surbl.org

1 participants
1867 discussions

problems when integrating SpamCopURI
by Arnold Nipper 26 Apr '04

26 Apr '04

Hi all, I just installed SpamCopURI (0.14) and restarted spamd. But now I get Apr 26 00:14:38 server spamd[6504]: Failed to compile URI SpamAssassin tests, skipping: (syntax error at /etc/mail/s pamassassin/spamcop_uri.cf, rule SPAMCOP_URI_RBL, line 1, near "eval:" syntax error at /usr/share/spamassassin/20_uri_tests. cf, rule E_MAILPROMO_URL, line 175, near "; }" ) Apr 26 00:14:39 server spamd[6504]: server started on port 783/tcp (running version 2.63) And further down a lot of lines like Apr 26 00:16:10 server spamd[16954]: Subroutine HTTP_ESCAPED_HOST_uri_test redefined at /usr/share/spamassassin/20_uri_tests .cf, rule HTTP_ESCAPED_HOST, line 5. Apr 26 00:16:10 server spamd[16954]: Subroutine BARGAIN_URL_uri_test redefined at /usr/share/spamassassin/20_uri_tests.cf, r ule BARGAIN_URL, line 8. Apr 26 00:16:10 server spamd[16954]: Subroutine URI_OFFERS_uri_test redefined at /usr/share/spamassassin/20_uri_tests.cf, ru le URI_OFFERS, line 8. ... So what's going wrong? Arnold

2 1

second and third level domains - again!
by John Fawcett 26 Apr '04

26 Apr '04

One of the things I noticed after upgrading to SpamCopURI 0.14 was that previously I had been identifying all mail containing ads.msn.com as spam and after the upgrade this was no longer happening. (BTW ads.msn.com was still listed in sc.surbl.org when I observed this behaviour. It has since been removed.) Version 0.14 includes the changes which were being discussed last week, so that if ads.msn.com is found in an email only msn.com is being checked against sc.surbl.org. So the choices available to the list maintainer are either: - list all of msn.com - list none of msn.com Since listing all of msn.com is likely to be too wide, this means msn.com will not get listed even if there are subdomains which are candidates for listing. I've used msn as an example, but the same logic applies to any of the big names like yahoo etc where the list maintainer may want to have more granularity in what is listed rather than list the whole registered domain. The solution could be to use a special return code which indicates "query again with more detail". (I remember someone bringing up something similar in the context of ccTLDs as well). So if ads.msn.com were to be listed in sc.surbl.org it would need two records: msn.com IN A 127.0.0.255 ads.msn.com IN A 127.0.0.2 The client (in this case SpamCopURI) would find a url ads.msn.com in the email but would query for msn.com as per the current logic. The return value of 127.0.0.255 then indicates to the client to query for one level lower, ie ads.msn.com. This same mechanism could be used for ccTLDs. sc.surbl.org could contain: co.uk IN A 127.0.0.255 co.nz IN A 127.0.0.255 So that if I get xxxxxxxx.co.uk in an email, the client queries for co.uk and it will be told to query with the lower level. The client queries for xxxxxxxx.co.uk The disadvantage of this solution is that it needs a changed logic on the client side, however the client does not need to know anything about which ccTLDs have two or three level domains. The extra query overhead is minimal, since on a busy server these additional records would almost always be in the DNS resolver cache. Even clients which don't implement this extra logic would however have to be careful to check the value of the A record rather than deduce the listing from the presence of an A record. Maybe we're still in time to make changes like this before there is too much software to change (currently I think there are 3). John

5 10

unique IPs in the blacklist
by Robert Brooks 26 Apr '04

26 Apr '04

how many unique IPs are there in the spamcop blacklist? has anyone considered creating a blacklist of the IP addresses of the hosts in the url blacklist? I'm guessing that the spammers are stuffing their sites on a relatively small number of webhosts. Regards, Rob -- Robert Brooks, Network Manager, Hyperlink Interactive Ltd <robb(a)hyperlink-interactive.co.uk> http://hyperlink-interactive.co.uk/ Tel: +44 (0)20 7240 8121 Fax: +44 (0)20 7240 8098 - Help Microsoft stamp out piracy. Give Linux to a friend today! -

2 1

packaging SpamCopURI
by Robert Brooks 26 Apr '04

26 Apr '04

Hi, I've packaged SpamCopURI, but these files clash with SpamAssassin files. /usr/lib/perl5/site_perl/5.6.1/Mail/SpamAssassin/Conf.pm /usr/lib/perl5/site_perl/5.6.1/Mail/SpamAssassin/PerMsgStatus.pm I assume that these normally replace the originally install SpamAssassin files. Is this correct, and if so should I look at adding SpamCopURI to my SpamAssassin package. Regards, Rob -- Robert Brooks, Network Manager, Hyperlink Interactive Ltd <robb(a)hyperlink-interactive.co.uk> http://hyperlink-interactive.co.uk/ Tel: +44 (0)20 7240 8121 Fax: +44 (0)20 7240 8098 - Help Microsoft stamp out piracy. Give Linux to a friend today! -

4 11

[SURBL-Discuss] Re: [spa] Re: Getting SURBL to deal with obfuscated URLs
by Jeff Chan 25 Apr '04

25 Apr '04

On Sunday, April 25, 2004, 11:55:33 AM, Charles Gregory wrote: > In the case of the '.ca' TLD (www.circa.ca), the original hierarchical > structure was: domain.city.prov.ca (eg fleanet.hamilton.on.ca). > The 'ca' registry has since been opened up and many new '.ca' domains > have the normal 2nd level format. But the registry still permits 3rd and > 4th level domains under the exsiting provincial and municipality levels. > I have no idea what safegaurds are in place, if any, to prevent spammers > from trying to register fourth level domains. I expect all they would have > to do is have a registered place of business in the municipality. > I do know that the named province/municipality does *not* have any control > over the assignments. Both two and three level domain cases are handled where the two level domains can be anything.ca that are not listed as a known geographic TLD like on.ca (for Ontario province) and where the three level domain would be under a known two level TLD such as anything.on.ca. A fourth level geographic domain, for example under a city name, may not be handled properly, but could be added. If we drive spammers to be desperate enough to use those then we've already had great success. :-) Jeff C. -- Jeff Chan mailto:jeffc@surbl.org http://www.surbl.org/

1 0

More news on unparsed urls
by John Fawcett 25 Apr '04

25 Apr '04

I have tested SpamCopURI 0.14 and SA 2.63 with my collection of unparsed urls. This new version deals with many of the cases, so that the ugly workarounds I was using can be removed. By the way, if you're reading this Eric, it might be worthwhile adding ads.msn.com and g.msn.com to the list of known redirection services in the sample spamcop_uri.cf. Here are the cases that are not picked up: 1. URLs that aren't URLs (missing protocol, even missing www ) <p> P<advisory>l<aboveboard>e<compose>a<geochronology>s<moral>e<palfrey> <rada= r>c<symptomatic>o<yankee>p<conduit>y<souffle> <intake>a<arise>n<eocene>d <= thickish>paste <impact>this <broadloom>link <road>i<dichotomous>n<quinine>= t<scoreboard>o y<eager>o<impact>ur b<archenemy>r<band>o<wallop>wser <b> he= althyexchange.biz</b> 2. Double protocol http://http://www.eager-18.com/_7953f10b575a18d044cdec5a40bd4f22//?d=vision Workaround in PerMsgStatus.pm $uri =~ s/http:\/\/http:\/\//http:\/\//gi; (NB from the previously published workaround I added case insensitivity) 3. HTML escape sequences in URL http://toform.net/mcp/879/1352/cap112.html Workaround in PerMsgStatus.pm $_ = HTML::Entities::decode($_); use HTML::Entities; (NB from the previously published workaround this is different because it does the conversion earlier on and so takes into account that http could also be coded with escape sequences. It seems to work despite the comment to not modify $_ in get_uri_list.) Here's a diff of PerMSgStatus.pm with SpamCopURI 0.14 compared to the version with the workarounds mentioned above. John diff -u PerMsgStatus.pm.orig PerMsgStatus.pm -----------cut------------- --- PerMsgStatus.pm.orig 2004-04-25 12:50:05.000000000 +0200 +++ PerMsgStatus.pm 2004-04-25 13:01:11.000000000 +0200 @@ -44,6 +44,7 @@ use Mail::SpamAssassin::Conf; use Mail::SpamAssassin::Received; use Mail::SpamAssassin::Util; +use HTML::Entities; use constant HAS_MIME_BASE64 => eval { require MIME::Base64; }; @@ -1748,6 +1749,7 @@ for (@$textary) { # NOTE: do not modify $_ in this loop + $_ = HTML::Entities::decode($_); while (/($uriRe)/go) { my $uri = $1; @@ -1776,6 +1778,7 @@ $uri = "${base_uri}$uri"; } } + $uri =~ s/http:\/\/http:\/\//http:\/\//gi; # warn("Got URI: $uri\n"); push @uris, $uri; -----------------cut---------------

1 0

domains that roll out of sc.surbl.org
by John Fawcett 24 Apr '04

24 Apr '04

Just a thought: given the dynamic nature of the sc.surbl.org data, it might be useful to have a listing history for the SURBL+ checker (http://www.rulesemporium.com/cgi-bin/uribl.cgi) Something along the lines of domain xxxx currently (listed|notl listed) in sc.surbl.org - added ddmmyyyy hhmm - removed ddmmyyyy hhmm - added ddmmyyyy hhmm - removed ddmmyyyy hhmm This way it would be possible to check whether old domains are being left dormant until they work their way out of the list and then being resurrected later on. Another thought was to make a separate historic list of domains that are removed from sc.surbl.org when there are no new reports. People who want to do more aggressive filtering could also use the historic list in addition to the other lists. The risk of FPs is still low because I can't forsee there being many "reformed" spammer domains. John

2 1

RE: [RulesEmporium] Rbldns and wildcards
by Dallas L. Engelken 24 Apr '04

24 Apr '04

> Indeed, I just realized an embarrassing screw-up on my part. > > There are two different programs with similar names and > slightly different properties,including zone file differences: > > rbldns > > rbldnsd > > We are using rbldnsd zone files. I need to update the web site > to reflect this and post an announcement. > As am I.. I guess I didn't realize there was a rbldns either... In rbldnsd, you can wildcard domains... From # man rbldnsd dnset Set of (possible wildcarded) domain names with associated A and TXT values. Similar to ip4set, but instead of IP addresses, data consists of domain names (not in reverse form). One domain name per line, possible starting with wildcard (either with starâdot (*.) or just a dot). Entry starting with exclamation sign is exclusion. Default value for all subsequent lines may be specified by a line starting with a colon. Wildcards are interpreted as follows: example.com only example.com domain is listed, not subdomains thereof. Not a wildcard entry. *.example.com all subdomains of example.com are listed, but not examâ ple.com itself. .example.com all subdomains of example.com and example.com itself are listed. This is a shortcut: to list a domain name itself and all itâs subdomains, one may either specify two lines (example.com and *.example.com), or one line (.examâ ple.com). Instead of listing FQDN's, why not just list the TLD with a dot in front... For example, list .mailnotice.biz instead of t.mailnotice.biz in case they change to some other letter in front of their TLD. Dallas

2 1

rbldnsd and rsync'ing
by Roger WJ Alterskjær 24 Apr '04

24 Apr '04

Hi all! I've successfully installed rbldnsd and want to set it up along with rsync'ing of sc/ws/be.surbl.org. My question is how to set up the RBLDNSD variable in /etc/sysconfig/rbldnsd and the rsync crontab. So far I've got a tip on rsync: Rsyncing is very easy. You would set up a cron job to call something like: rsync RSYNC_SERVER_NAME_GOES_HERE::surbl/sc.surbl.org.rbldns . Seams simple enough. It's just that darn rbldnsd variable that's got me confused. -- Mvh, Roger WJ Alterskjær Edb-konsulent Vitenskapsmuseet, NTNU

1 0

Fwd: Re: Goofy domain names
by Jeff Chan 24 Apr '04

24 Apr '04

From: Jeff Chan To: spamassassin-users Date: Friday, April 23, 2004, 3:28:49 PM Subject: Goofy domain names ===8<==============Original message text=============== On Friday, April 23, 2004, 7:38:46 AM, Chris Santerre wrote: > This is where BigEvil may start going. I can change mine in 2 secs to use > /\d00\dhosting/ but as soon as I do that, it will be removed from > be.surbl.org. For obvious reasons they can't use wildcards. All signs point > to me changing bigevil over to search for this kind of stuff, and simply add > any static ones I have to ws.surbl.org. But will see. This is where our philosophies clash slightly. SURBLs just want a list of known spam domains. SA rulesets with wildcards try to match entire possible/probable classes of domain names based on observing prior types of variation. Both approaches have their merits. For my purposes, I'd just prefer to get the domains that have already been found in spam. I acknowledge that that doesn't have the predictive value of the class approach, but it also makes FPs less possible in principle. (Though in reality it's not very likely that any legitimate sites are suddenly going to start using rxmeds1.com, rxmeds2.com, rxmeds3.com, etc.) Jeff C. ===8<===========End of original message text===========

3 4

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Discuss