Discuss

discuss@lists.surbl.org

2 participants
1870 discussions

Re: [SURBL-Discuss] Simon's complex redirection
by Patrik Nilsson 28 Apr '04

28 Apr '04

At 10:55 2004-04-28 +1200, Simon Byrnand wrote: > > It's double-encoded. We can catch that easily. But first, my question -- > > does this *work* in an MUA, ie. should we? Simon, could you try it? > >What you get is the image preview in google which consists of an image in >the top frame, and the page that it came from in the bottom frame, and in >the bottom frame was a link "click here for ......." so yes it definately >does work... I guess this is a 'framer' rather than a redirector. A url that points to a frameset that loads an external page specified in the query url in one of the frames. For all practical purposes it's similar to a redirector, but with complications. The http response code from a GET request using the url will not indidate that it's a redirector. For a user loading the page in a browser, the adress of the spamvertized website will not be visible in the location bar. For some spammers, this would probably be prefered over traditional redirectors. Patrik

2 1

Re: Simon's complex redirection
by jm＠jmason.org 28 Apr '04

28 Apr '04

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Theo Van Dinter writes: > On Tue, Apr 27, 2004 at 03:46:32PM -0700, Justin Mason wrote: > > > http://images.google.ca/imgres?imgurl=gmib.free.fr/viagra.jpg&imgrefurl… > > eek! > > > It's double-encoded. We can catch that easily. But first, my question -- > > does this *work* in an MUA, ie. should we? Simon, could you try it? > > I don't know about an MUA, but it doesn't work in any browser I've tried > so far. You end up at http://images.google.com/images: > > Information returned from request (not including data): > <Connection> = <Keep-Alive> > <Content-Type> = <text/html> > <Content-length> = <161> > <Date> = <Tue, 27 Apr 2004 22:59:47 GMT> > <Location> = <http://images.google.com/images> > <Server> = <GWS/2.1> > <Set-Cookie> = <PREF=ID=2342b69354b6a6f5:LD=en:TM=1083106787:LM=1083106787:S=4Q5Kb3zcm_6V0Tjm; expires=Sun, > 17-Jan-2038 19:14:07 GMT; path=/; domain=.google.ca> > <code> = <301> Try http://images.google.ca/imgres?imgurl=gmib.free.fr/viagra.jpg&imgrefurl=htt… (NOTE: with & replaced with just &) that works. (although it still uses google's image results page for the display.) But just because it ends up at the imgres page doesn't mean we don't have to worry about it. This URL: http://www.google.com/url?q=http://www.google.com/url?q=%68%74%74%70%3A%2F%… also works -- Google redirects it to the target site http://www.expage.com/manger32 successfully -- but SpamAssassin doesn't parse it. - --j. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Exmh CVS iD8DBQFAjurMQTcbUG5Y7woRAi2KAKDkGfoSzXWzzP73ggTr0FEZYlbEEgCg7861 JrRcI4laMJDWumLpHAbWaT4= =/Hlu -----END PGP SIGNATURE-----

1 0

RE: [SURBL-Discuss] Fwd: Re: Goofy domain names
by Chris Santerre 26 Apr '04

26 Apr '04

>-----Original Message----- >From: Mark [mailto:admin@asarian-host.net] >Sent: Saturday, April 24, 2004 12:11 AM >To: David B Funk; SURBL Discussion list >Cc: SpamAssassin Users >Subject: Re: [SURBL-Discuss] Fwd: Re: Goofy domain names > > >David B Funk wrote: > >> On Sat, 24 Apr 2004, Mark wrote: >> >>> Why use expand_regex.pl at all? Instead of taking "/\d00\dhosting/", >>> and expand it into 100 potential domain names, why not simply accept >>> a query like: >>> >>> 9001hosting.com.be.surbl.org >>> >>> And, internally, do the regex "/\d00\dhosting/" on it? (and all >>> BigEvil regexes, for that matter). And return 127.0.0.2 on a match. >>> >>> Ultimately, using expand_regex.pl, imho, is utterly >self-defeating -- >>> without stretching the imagination too much, even. If only I change >>> the above regex to: >>> >>> /\d+00\d+hosting/ >>> >>> We're already looking at trillions of permutations! >>> >>> It seems to me be.surbl.org should not do an internal database query >>> on single domain names, but instead do a BigEvil regex (like SA >>> does). At least, that is what I thought it would do; and how I had >>> hoped it would have been. >> >> Because the DNS system is designed to work with static lists of data, >> not regex patterns. >> >> Theoretically you could design a DNS server front-end on a regex >> engine that would take fixed DNS queries and do regex matches. >> However the DNS zone-transfer mechanism is designed to deal with >> fixed data-sets, not regexes. > >DNS zone-transfers would not be affected. It would still be >the same, static >data-set being pushed back and forth. In fact, the >be.surbl.org DNS server >would not contain a regular DNS database at all (!), as it merely does >regexes on incoming queries! If anything, you'd have to transer BigEvil >regex rules, really. > >> Another detail, is that the DNS system is heavily based upon the >> concept of cacheing. Currently there's no meaningful way to cache a >> regex, only the results of every lookup permutation. That would >> impose an unrealistic demand on the down-stream DNS servers (remember >> those trillions of permutations!), > >Not at the client-side. When I were to query >"9001hosting.com.be.surbl.org", >there would be no overhead whatsoever, cache-wise, for a front-end DNS >system that does a behind-the-scenes regex on the query. It would still >reply with a single response. It would, to the client, appear >no different >then if the DNS server had retrieved the domain name from database. > >The caching DNS server at be.surbl.org itself might grow big, >though, as it >potentially holds those trillions of those domain names. But >it would, as >you say, really only cache the result of each looked-up >permutation. And no >more. And, in this, it would be no different from a regular DNS server. > >> Another detail, you're adding to the CPU load on the "regex-ifed" DNS >> servers. Doing a regex match is more CPU intensive than a database >> lookup on a fixed data set. When you have the potential of thousands >> or millions of clients beating upon a handfull of DNS servers, the >> CPU load considerations become critical. > >This is true. And it may indeed be quite a problem. > >> Might as well stick with BigEvil.cf > >My point exactly; without using real regexes in be.surbl.org, dropping >BigEvil.cf, for now, is ill-advised, really. > >Cheers! > >- Mark Wow that was a lot to read for a Monday Morning :) Bigevil won't be dropped. But it will change. You will have lots of options in the near future. Lots of options ;) --Chris

1 0

problems when integrating SpamCopURI
by Arnold Nipper 26 Apr '04

26 Apr '04

Hi all, I just installed SpamCopURI (0.14) and restarted spamd. But now I get Apr 26 00:14:38 server spamd[6504]: Failed to compile URI SpamAssassin tests, skipping: (syntax error at /etc/mail/s pamassassin/spamcop_uri.cf, rule SPAMCOP_URI_RBL, line 1, near "eval:" syntax error at /usr/share/spamassassin/20_uri_tests. cf, rule E_MAILPROMO_URL, line 175, near "; }" ) Apr 26 00:14:39 server spamd[6504]: server started on port 783/tcp (running version 2.63) And further down a lot of lines like Apr 26 00:16:10 server spamd[16954]: Subroutine HTTP_ESCAPED_HOST_uri_test redefined at /usr/share/spamassassin/20_uri_tests .cf, rule HTTP_ESCAPED_HOST, line 5. Apr 26 00:16:10 server spamd[16954]: Subroutine BARGAIN_URL_uri_test redefined at /usr/share/spamassassin/20_uri_tests.cf, r ule BARGAIN_URL, line 8. Apr 26 00:16:10 server spamd[16954]: Subroutine URI_OFFERS_uri_test redefined at /usr/share/spamassassin/20_uri_tests.cf, ru le URI_OFFERS, line 8. ... So what's going wrong? Arnold

2 1

second and third level domains - again!
by John Fawcett 26 Apr '04

26 Apr '04

One of the things I noticed after upgrading to SpamCopURI 0.14 was that previously I had been identifying all mail containing ads.msn.com as spam and after the upgrade this was no longer happening. (BTW ads.msn.com was still listed in sc.surbl.org when I observed this behaviour. It has since been removed.) Version 0.14 includes the changes which were being discussed last week, so that if ads.msn.com is found in an email only msn.com is being checked against sc.surbl.org. So the choices available to the list maintainer are either: - list all of msn.com - list none of msn.com Since listing all of msn.com is likely to be too wide, this means msn.com will not get listed even if there are subdomains which are candidates for listing. I've used msn as an example, but the same logic applies to any of the big names like yahoo etc where the list maintainer may want to have more granularity in what is listed rather than list the whole registered domain. The solution could be to use a special return code which indicates "query again with more detail". (I remember someone bringing up something similar in the context of ccTLDs as well). So if ads.msn.com were to be listed in sc.surbl.org it would need two records: msn.com IN A 127.0.0.255 ads.msn.com IN A 127.0.0.2 The client (in this case SpamCopURI) would find a url ads.msn.com in the email but would query for msn.com as per the current logic. The return value of 127.0.0.255 then indicates to the client to query for one level lower, ie ads.msn.com. This same mechanism could be used for ccTLDs. sc.surbl.org could contain: co.uk IN A 127.0.0.255 co.nz IN A 127.0.0.255 So that if I get xxxxxxxx.co.uk in an email, the client queries for co.uk and it will be told to query with the lower level. The client queries for xxxxxxxx.co.uk The disadvantage of this solution is that it needs a changed logic on the client side, however the client does not need to know anything about which ccTLDs have two or three level domains. The extra query overhead is minimal, since on a busy server these additional records would almost always be in the DNS resolver cache. Even clients which don't implement this extra logic would however have to be careful to check the value of the A record rather than deduce the listing from the presence of an A record. Maybe we're still in time to make changes like this before there is too much software to change (currently I think there are 3). John

5 10

unique IPs in the blacklist
by Robert Brooks 26 Apr '04

26 Apr '04

how many unique IPs are there in the spamcop blacklist? has anyone considered creating a blacklist of the IP addresses of the hosts in the url blacklist? I'm guessing that the spammers are stuffing their sites on a relatively small number of webhosts. Regards, Rob -- Robert Brooks, Network Manager, Hyperlink Interactive Ltd <robb(a)hyperlink-interactive.co.uk> http://hyperlink-interactive.co.uk/ Tel: +44 (0)20 7240 8121 Fax: +44 (0)20 7240 8098 - Help Microsoft stamp out piracy. Give Linux to a friend today! -

2 1

packaging SpamCopURI
by Robert Brooks 26 Apr '04

26 Apr '04

Hi, I've packaged SpamCopURI, but these files clash with SpamAssassin files. /usr/lib/perl5/site_perl/5.6.1/Mail/SpamAssassin/Conf.pm /usr/lib/perl5/site_perl/5.6.1/Mail/SpamAssassin/PerMsgStatus.pm I assume that these normally replace the originally install SpamAssassin files. Is this correct, and if so should I look at adding SpamCopURI to my SpamAssassin package. Regards, Rob -- Robert Brooks, Network Manager, Hyperlink Interactive Ltd <robb(a)hyperlink-interactive.co.uk> http://hyperlink-interactive.co.uk/ Tel: +44 (0)20 7240 8121 Fax: +44 (0)20 7240 8098 - Help Microsoft stamp out piracy. Give Linux to a friend today! -

4 11

[SURBL-Discuss] Re: [spa] Re: Getting SURBL to deal with obfuscated URLs
by Jeff Chan 25 Apr '04

25 Apr '04

On Sunday, April 25, 2004, 11:55:33 AM, Charles Gregory wrote: > In the case of the '.ca' TLD (www.circa.ca), the original hierarchical > structure was: domain.city.prov.ca (eg fleanet.hamilton.on.ca). > The 'ca' registry has since been opened up and many new '.ca' domains > have the normal 2nd level format. But the registry still permits 3rd and > 4th level domains under the exsiting provincial and municipality levels. > I have no idea what safegaurds are in place, if any, to prevent spammers > from trying to register fourth level domains. I expect all they would have > to do is have a registered place of business in the municipality. > I do know that the named province/municipality does *not* have any control > over the assignments. Both two and three level domain cases are handled where the two level domains can be anything.ca that are not listed as a known geographic TLD like on.ca (for Ontario province) and where the three level domain would be under a known two level TLD such as anything.on.ca. A fourth level geographic domain, for example under a city name, may not be handled properly, but could be added. If we drive spammers to be desperate enough to use those then we've already had great success. :-) Jeff C. -- Jeff Chan mailto:jeffc@surbl.org http://www.surbl.org/

1 0

More news on unparsed urls
by John Fawcett 25 Apr '04

25 Apr '04

I have tested SpamCopURI 0.14 and SA 2.63 with my collection of unparsed urls. This new version deals with many of the cases, so that the ugly workarounds I was using can be removed. By the way, if you're reading this Eric, it might be worthwhile adding ads.msn.com and g.msn.com to the list of known redirection services in the sample spamcop_uri.cf. Here are the cases that are not picked up: 1. URLs that aren't URLs (missing protocol, even missing www ) <p> P<advisory>l<aboveboard>e<compose>a<geochronology>s<moral>e<palfrey> <rada= r>c<symptomatic>o<yankee>p<conduit>y<souffle> <intake>a<arise>n<eocene>d <= thickish>paste <impact>this <broadloom>link <road>i<dichotomous>n<quinine>= t<scoreboard>o y<eager>o<impact>ur b<archenemy>r<band>o<wallop>wser <b> he= althyexchange.biz</b> 2. Double protocol http://http://www.eager-18.com/_7953f10b575a18d044cdec5a40bd4f22//?d=vision Workaround in PerMsgStatus.pm $uri =~ s/http:\/\/http:\/\//http:\/\//gi; (NB from the previously published workaround I added case insensitivity) 3. HTML escape sequences in URL http://toform.net/mcp/879/1352/cap112.html Workaround in PerMsgStatus.pm $_ = HTML::Entities::decode($_); use HTML::Entities; (NB from the previously published workaround this is different because it does the conversion earlier on and so takes into account that http could also be coded with escape sequences. It seems to work despite the comment to not modify $_ in get_uri_list.) Here's a diff of PerMSgStatus.pm with SpamCopURI 0.14 compared to the version with the workarounds mentioned above. John diff -u PerMsgStatus.pm.orig PerMsgStatus.pm -----------cut------------- --- PerMsgStatus.pm.orig 2004-04-25 12:50:05.000000000 +0200 +++ PerMsgStatus.pm 2004-04-25 13:01:11.000000000 +0200 @@ -44,6 +44,7 @@ use Mail::SpamAssassin::Conf; use Mail::SpamAssassin::Received; use Mail::SpamAssassin::Util; +use HTML::Entities; use constant HAS_MIME_BASE64 => eval { require MIME::Base64; }; @@ -1748,6 +1749,7 @@ for (@$textary) { # NOTE: do not modify $_ in this loop + $_ = HTML::Entities::decode($_); while (/($uriRe)/go) { my $uri = $1; @@ -1776,6 +1778,7 @@ $uri = "${base_uri}$uri"; } } + $uri =~ s/http:\/\/http:\/\//http:\/\//gi; # warn("Got URI: $uri\n"); push @uris, $uri; -----------------cut---------------

1 0

domains that roll out of sc.surbl.org
by John Fawcett 24 Apr '04

24 Apr '04

Just a thought: given the dynamic nature of the sc.surbl.org data, it might be useful to have a listing history for the SURBL+ checker (http://www.rulesemporium.com/cgi-bin/uribl.cgi) Something along the lines of domain xxxx currently (listed|notl listed) in sc.surbl.org - added ddmmyyyy hhmm - removed ddmmyyyy hhmm - added ddmmyyyy hhmm - removed ddmmyyyy hhmm This way it would be possible to check whether old domains are being left dormant until they work their way out of the list and then being resurrected later on. Another thought was to make a separate historic list of domains that are removed from sc.surbl.org when there are no new reports. People who want to do more aggressive filtering could also use the historic list in addition to the other lists. The risk of FPs is still low because I can't forsee there being many "reformed" spammer domains. John

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Discuss