Discuss September 2004

discuss@lists.surbl.org

62 participants
229 discussions

Re: [SURBL-Discuss] "Unconfirmed" list proof-of-concept
by Rob McEwen 03 Sep '04

03 Sep '04

> If Ryan (or someone else) wants to take submissions directly, for > different lists, fine, but do not use the mailing list for this purpose, > the volume is high enough as it is now. > > Again, this mail is not trying to upset anyone, i think most of you will > understand our point of view, if not we would be willing to point that out > offlist. Even thought I vote FOR unconfirmed.surbl.org, I'd rather wait and let this issue work itself out than rush ahead and, as Jeff put it, "divide our efforts". Therefore, I have some constructive suggestions for how to unify our efforts in the meantime towards things I think we can all agree on which will only help achieve ALL of our goals, **even the ones that we disagree on :)** (I'll explain how...) FIRST: While we are waiting to see (for sure) what happens with unconfirmed.surbl.org, there are other ways to do greylisting. Please allow me to share some of the things that I do. (forgive me that some of these are unconventional and are not "by the book"... no lectures please). **I convert domains to IP addresses both in the body and headers and check these IP addresses and other IP address in the message against SpamHaus (they even suggest doing this in their literature... at least to some extent). **I check header URIs against SURBL (again, no lectures please... I do this more for auditing than for actual blocking) NOTE...I have actually found that these first two generate a high rate of spam with few false positives.... but there ARE enough false positives to keep me from using this for blocking in a "set it and forget it" mode, but... a high amount of spam "hits" nevertheless. **After my SpamHaus/DSBL MTA blocking is done, and after my linguistic filter & SURBL filtering is done... I then check the sending IP of each message still not blocked against both psbl.surriel.com and bl.spamcop.net . A lot of legitimate messages get "flagged" here... but still a rich auditing pool. This helps me quickly find much of what is getting past my filter without having to constantly dig through my client's e-mail. ...ALSO, in all these cases, I create bypass lists for each level of checking to prevent false "hits" on each of these checks so that I'm not constantly looking at the same stuff over and over again. I also blacklist obvious URIs in my linguistic filter that I deem not possibly ever legitimate. (I should probably be submitting these to SURBL? ... I have some follow questions on that process for another tread.) Looking at the data and what actually happens... my SpamHaus technique **IS** almost like what I think that people are looking for in unconfirmed.surbl.org SECOND: 5 days ago, I had a suggestion regarding integrating tracking of these Greymarketer organizations into the upcoming formal SURBL submission and tracking system we've talked about. The basic idea was to find a way to put pressure on them, generate an awareness of these groups, and try to help each other develop tools for blocking their spams without blocking the legitimate messages. Changing the subject slightly, another idea behind this was to work together to develop strategies for blocking spam from those proliferate spammers who often beat SURBL for a few hours or days with their new domains. For example, recall that "pesky porn spam" request I made last month... one of the guys on this list (forgot his name) gave me some suggestions and I develop a rule in my filter which catches ALL of these with zero chance of false positives. FINALLY: Overall, I suggest that there are plenty of things we can work on... even "out of the box" things... that we can all agree on. I suggest we work on these and shelve "unconfirmed.surbl.org" in the meantime. A good compromise is to take my suggestion and integrate tracking of things that would be good candidates for the UC list a a formal category in the format SURBL tracking system that we all agreed should be created. That way, if we later DO formally decide to do the UC list, the pro-UC folks will know that they are already further along in that task than we are now. Thoughts? BELOW IS MY EARLIER MESSAGE FROM ABOUT A WEEK AGO THAT I REFERRED TO: I think that there are two areas which present particular challenges. (1) e-mail marketers who play it "both ways"... thus making it hard to use SURBL to catch their bad behavior without blocking legitimate mail ...AND... (2) savy spammers who manage to get significant amounts through in the first few minutes/hours BEFORE getting blocked by SURBL... in particular, the ones who already use the best strategies to get around all other types of filtering. The quicker TTLs is helping with the savvy spammers. Also, I recall something about a newer version of SURBL which will use some kind of tracking to trace new domains back to older ones in order to attributing new spam to known and confirmed spammers so that they would stay "attached" to their previous bad records in order to blacklist them faster. What ever became of this? (Did I explain this correctly?) Anyway... even when these are done, we will STILL have some problems with the most savvy spammers. Also, I think that a lot of people fear that, as we work towards eliminating the rest of the FPs, more and more spam from these e-mail marketers who play it "both ways" will get through and the overall catch rate for SURBL may drop by 10 or 20 percent (or whatever). I'm willing to live with that... (gulp!) BUT... I think that it would be great to integrate into a formal tracking system a way to categorize URIs into either or both of these groups. ("SavySpams" and "GrayMarketer" ...or whatever) That way, we can use this data to help us form better "rules" in our linguistic/heuristic filters. The idea being that, at this point, the amount of spam that is getting through is much more focused than a large general pool of spam. This more narrow focus should give us the tools to close any loopholes that SURBL might not catch. I would also suggest that if a message's server address is already blocked by BOTH list.dsbl.org AND sbl-xbl.spamhaus.org, then it shouldn't be added to this particular list for the sake of keeping the list focused. It seems that, whatever the disagreements about RLBs are, I think that EVERYONE would agree with this particular standard as being a reliable (yet FP safe and conservative) standard for RBL blocking. I envision a "Gray page" which would list the top 10 offenders of Graymarketers who are bad enough to be mentioned, but not bad enough to get listed by SURBL and the top 10 "savvy spammers" who are known to periodically (abet temporarily) beat SURBL with their new domains. Subsequent offenders could be listed on following pages after the top 10 for each of these two categories. Each listing would include a link to more info about this spammer or series of spam. This more info page would also included samples of spam that hit real spamtraps (made anonymous), and, for the gray marketers, samples of legitimate mail with that particular URI. FP-safe rules would also be suggested... NOW... who would have the time to get all this together??? :) Rob McEwen

1 0

FP: autoscout24.xentranmailMUNGED.com
by Alex Broens 03 Sep '04

03 Sep '04

FP: autoscout24.xentranmailMUNGED.com * 5.0 OB_URI_RBL URI's domain appears in ws database at ob.surbl.org * [autoscout24.xentranmailMUNGED.com is blacklisted in] [URI RBL at multi.surbl.org] http://www.autoscout24.ch/ AutoScout24 AG Bernstrasse 41 3175 Flamatt E-Mail: info(a)autoscout24.ch Web: http://www.autoscout24.ch WHOIS: http://www.switch.ch/search/whois_form.html?Query=autoscout24.ch&Server=who… very white hat Pls whitelist & remove ASAP. thanks Alex

2 4

RE: [SURBL-Discuss] Proposing a greylist
by Chris Santerre 03 Sep '04

03 Sep '04

WOW I can't believe the analysis paralysis we got going on here! I say we don't add anything to the unconfirmed (I'm not calling it grey anymore) list until we get at least 2-3 people on this list to agree to. No one person should add to unconfirmed. We end up discussing these anyway when we check for blacklisting. So we will already be discussing. I see the unconfirmed list being very small for now. We don't put it in multi for now. I'm still doing all the work to check for blacklist submission, but I realize they have the possibility of having some small % of legit use. Like less then 20%. Then I'm asking the list to vote for unconfirmed. Simple. Otherwise if it is 50% ham/spam, it doesn't get listed even in unconfirmed. Simple as that. No extra work. We try here just in this list. Don't make it public yet. This changes absolutely nothing I don't already do for SURBL. Just asks for 2 others to agree with me, which I already do for "iffy" domains. This will also change submissions making it EASIER for us. People who do a SURBL lookup would see it has ALREADY been submitted and has been found to be unconfirmed by the SURBL team. One less submission and one less person saying "Why haven't you added them?" Who said they were up for hosting this? Because I'm ready to do this alone if I have to. There is a need, even if we only use it internally for research and not make it public, I need to know some of this info. --Chris (Sometimes you just have to skate hard into the corners and worry about the outcome later.)

2 1

RE: [SURBL-Discuss] RE: Applying SURBL against blog comment spamm ers
by Chris Santerre 03 Sep '04

03 Sep '04

>-----Original Message----- >From: Jeff Chan [mailto:jeffc@surbl.org] >Sent: Thursday, September 02, 2004 9:23 PM >To: 'SURBL Discussion list' >Subject: Re: [SURBL-Discuss] RE: Applying SURBL against blog comment >spammers > > >On Thursday, September 2, 2004, 7:28:16 AM, Rob McEwen wrote: >(Jeff Chan wrote:) >>>Given the lack of commonality, it may not make much sense to >>>add to the mail spam lists, since it would be an extra 2000+ >>>records that would probably not get hits on mail. > >>>The MT-Blacklist doesn't seem to update too frequently (the >>>last new record was from 8/29) and has about 2000 records. >>>Matthew's list was pretty sparse so far. So I'm still >>>pondering things. > >> I could be totally wrong... but I suspect that the lack of >commonality may >> be more due to either the MT-blacklist not being updated as >frequently or >> because the MT-blacklist may not be updated as extensively. > >> A good test may be to "Google" some more frequently found >SURBL-blocked >> domains which are not on the MT-blacklist along with the >phrase "moveable >> type" ...you might find a lot of blog-comment spam which >should have been on >> the MT-blacklist and is already in SURBL. > >Hmm, interesting. Other comments also seem to suggest that the >blog spammers are sometimes the same as mail spammers. So maybe >there should be more overlap, but additions to MT-blacklist are >slower than SURBLs. > >On the other hand, our databases are pretty far reaching and >should have hit on even older blog spam domains, yet they >largely didn't. > >The quick and easy answer, which may be wrong, is that they're >different folks, or at least different domains. > >Jeff C. > Oh please don't think that just yet!! Seriously. I'm working with some ninjas and the 6dos data and a new tool to let you look up this info! So far it ROCKS beyond belief! But more coming, and trying to keep data source anonymous of course. Also trying to tie in some other tools that other SURBL submitters have been asking for. Bottom line is that these guys ARE the same people. Data shows it. ON the blog spam pointing to other blog spams, well I would LART the submitter who didn't hand check that! Just like the ones that try to submit ebay and junk! I follow up all of those, and would see it points to another blog. Then I'd follow that until I had a KNOWN spam domain to add. If I didn't have enough proof, I wouldn't list. I have a "Not added" folder that grows everyday because of this. :) --Chris (Way too many projects going on!!)

2 1

RE: Applying SURBL against blog comment spammers
by Chris Santerre 03 Sep '04

03 Sep '04

>-----Original Message----- >From: Jeff Chan [mailto:jeffc@surbl.org] >Sent: Thursday, September 02, 2004 9:04 PM >To: spamassassin-users(a)incubator.apache.org >Cc: SURBL Discuss >Subject: Re: Applying SURBL against blog comment spammers > > >On Thursday, September 2, 2004, 5:43:26 PM, Loren Wilton wrote: >>> Given the lack of commonality, it may not make much sense to >>> add to the mail spam lists, since it would be an extra 2000+ >>> records that would probably not get hits on mail. >>> >>> The MT-Blacklist doesn't seem to update too frequently (the >>> last new record was from 8/29) and has about 2000 records. >>> Matthew's list was pretty sparse so far. So I'm still >>> pondering things. > >> Just from a technical/philosophical point, I think a separate list is >> desirable. Although I agree that making it part of multi >would probably be >> the way to go, and I agree with the basic concept that "spam >is spam". > >> However, I think the reasons for a separate list are: > >> 1. Separate source feed. A new list allows the source >feed to be more >> easily documented. >> 2. (As stated) little overlap with email spammers, at >least so far. >> 3. Probably a different update cycle and removal (from >old age) cycle >> requirement > >> The different means of updating and possibly different aging >method are high >> on my list of reasons for suggesting a separate list. On >the other hand, >> having it part of multi would be nice, since (I assume, possibly >> incorrectly) that one query could check a lot of lists based >on the bitmap. > >Correct. I'm still wavering if a blog spam list should be part >of multi. There are programs that use multi but (unadvisedly) >don't differentiate between the source lists. That kind of >argues for keeping multi focussed on only mail spam and making >a blog spam list separate. On the other hand there's much less >overhead in adding a list internally to multi than setting up >a whole new list. > >> It probably would also be good to devote some thought to how >entries will be >> added to this list and validated. We surely don't want some >annoyed blog >> spammer spamming the list with every valid doamin they can find! > >Yes, data quality is always an issue. Any of these ventures will >struggle if spammers are able to poison the data. Keeping >legitimate domains out of any feed is key and provisions would >need to be made for that. > As always I agree. I think any new idea should be kept out of mutli until more testing is done. Having said that, if a blog spam matches all the requirments we would use in a SURBL entry now, then why not list? And why not list in the regular WS list? I'm saying I would only add hand checked domains like those I found in JM's example. Preemptive listings. Again, only domains that are obvious, like the examples I listed. If there is any question of a blog spam domain being used for legit, then it follows the very same rules we have now. Don't blacklist. Add to unclassified ;) --Chris

1 0

RE: [SURBL-Discuss] Whitelist Please
by Chris Santerre 03 Sep '04

03 Sep '04

>-----Original Message----- >From: Jeff Chan [mailto:jeffc@surbl.org] >Sent: Thursday, September 02, 2004 9:24 AM >To: SURBL Discuss >Subject: Re: [SURBL-Discuss] Whitelist Please > > >On Thursday, September 2, 2004, 6:19:04 AM, Chris Santerre wrote: >> Uh...I'm confused. So you did or did not whitelist all of >these. I would >> have thought that this one link would have been enough. Confused :) > >They are all whitelisted. > >> Flogo sucks donkey doughnuts. HOWEVER there are lusers who >sign up for it. I >> would not list them. But some of these other domains, I would list. > >I'm going to assume similar policies across all their domains. >It doesn't make much sense for them to be tame spammer on some >sites and uber spammer on others. > >> euniverse is not squeeky clean. > >Never said they were. The question is do they have legitimate >uses, and it sounds like the answer is yes. > >Jeff C. > Did I ever send the RBL results for dietingplans.com? Results: Positive=9, Negative=21 (2004-09-01 19:09:48 UTC) * @SPAM/spamsource: 12.129.205/24: 553 SPAM,BULK ATT mail3051.flowgo.com special(a)intelligentx.com 2002-04; SBL http://www.spamhaus.org/SBL/sbl.lasso?query=SBL5535; SPEWS [1] euniverse/yourbigvote, see http://spews.org/ask.cgi?S1 * AHBL/dnsbl.ahbl.org: 12.129.205/24: 553 AHBL Spam Source; AHBL Open Proxy [Details] * AUDNSBL/dnsbl.net.au: 12.129.205/24: 553 AUDNSBL Multiple Spam Traps Block List [Remove] * DRBL/drbl.all: 12.129.205/24: 553 DRBL weight: 1.4; vote.drbl.sandy.ru(a)ns.sci-nnov.ru/0.6 vote.drbl.vsu.ru(a)ns.vsu.ru/0.8; flowgo.com; flowgo.com spam engine. 20020218. Blocked by blacklist-admin(a)vsu.ru * SBL/spamhaus.org: 12.129.205/24: 553 SBL http://www.spamhaus.org/SBL/sbl.lasso?query=SBL5535 * SPEWS/spews.org: 12.129.205/24: 553 SPEWS2 [1] euniverse/yourbigvote, see http://spews.org/ask.cgi?S1677 * BLARS/block.blars.org: INET 127.3.8.32 * INTERSIL/flowgo.bulk: Flowgo/Euniverse/media synergy/etc.; http://www.cauce.org/incidents/flonetwork.com/index.shtml; 2001May01; dig TXT netlist.flowgoaway.com for CIDR list * FIVETEN/flonetwork.com.bulk: UNKNOWN TXT (permfail,nodata) I mean, to me this is a no brainer to list. :) --Chris (I'm annoying sometimes!)

4 3

RE: Applying SURBL against blog comment spammers
by Chris Santerre 03 Sep '04

03 Sep '04

>-----Original Message----- >From: Jeff Chan [mailto:jeffc@surbl.org] >Sent: Thursday, September 02, 2004 3:24 AM >To: SATalk >Cc: SURBL Discuss >Subject: Re: Applying SURBL against blog comment spammers > > >On Wednesday, September 1, 2004, 11:25:40 PM, Matthew Hunter wrote: >> I just whipped up some code to reject trackback/comment spam >> using a SURBL as a data source. Unfortunately, the people >> spamming my weblogs aren't in multi.surbl.org, so I will have to >> maintain my own local blacklist server. > >> The single most useful thing that could be done wrt fighting spam >> in weblogs would be an SURBL source that had the offending >> domains in it. I would offer to make mine public, but I don't >> have the IP to spare at the moment... > >> Does anyone know of an appropriate SURBL list? > >Hi Matthew, >We could perhaps set up a separate SURBL for blog spammers. >It would be a slight shift in focus since the other SURBLs are >all for email spam. Can you give an idea of how many records >you have? > >Also have you tried Jay Allen's MT-Blacklist/Comment Spam >list: > > http://www.jayallen.org/comment_spam/ > >It would be interesting to look at your data to see if there's >much overlap with our existing lists. In the case of Jay's data, >there's nearly none. > Hell I'm feeling a little saucy this morning so lets mull this over. This goes against Jeff's thoughts. But if they are spamming, then just add them to SURBL. Does it matter if they spam email or blogs? To me, not really. Adding them to the regular SURBL is sure to cause them some pain. Legit domains still get removed. SO I say, go ahead and add them. However I would like to see an example of a spam'd blog. I've never seen one. --Chris

7 12

Re: RE: [SURBL-Discuss] Proposing a greylist
by Rob McEwen 03 Sep '04

03 Sep '04

> > 1)We DO NOT include it in multi. > > Please reconsider this... Including it in multi means a lot less DNS > traffic, and that's a serious plus when you're using a greylist that you > probably won't use to increase the spam score by much. > > Bret Bret, You are reinforcing all of Jeff's arguments for NOT doing greylisting. If we create an unconfirmed.surbl.org list and then merge this into the multi list, then we might as well not worry about FPs anymore. Heck, we might was well throw every domain found in every spamtrap and be done with spam forever ;) Also, the fact that you would even suggest this further proves Jeff's point. I'm all in favor of creating an unconfirmed.surbl.org list. But it would HAVE to stay separate from the multi.surbl.org list or ALL the problems Jeff described will come true. ...not to mention, this would also result in even MORE DNS traffic because many of use would then start using EVERY list but the multi list. Rob McEwen

2 1

Re: Applying SURBL against blog comment spammers
by Jeff Chan 03 Sep '04

03 Sep '04

On Thursday, September 2, 2004, 5:43:26 PM, Loren Wilton wrote: >> Given the lack of commonality, it may not make much sense to >> add to the mail spam lists, since it would be an extra 2000+ >> records that would probably not get hits on mail. >> >> The MT-Blacklist doesn't seem to update too frequently (the >> last new record was from 8/29) and has about 2000 records. >> Matthew's list was pretty sparse so far. So I'm still >> pondering things. > Just from a technical/philosophical point, I think a separate list is > desirable. Although I agree that making it part of multi would probably be > the way to go, and I agree with the basic concept that "spam is spam". > However, I think the reasons for a separate list are: > 1. Separate source feed. A new list allows the source feed to be more > easily documented. > 2. (As stated) little overlap with email spammers, at least so far. > 3. Probably a different update cycle and removal (from old age) cycle > requirement > The different means of updating and possibly different aging method are high > on my list of reasons for suggesting a separate list. On the other hand, > having it part of multi would be nice, since (I assume, possibly > incorrectly) that one query could check a lot of lists based on the bitmap. Correct. I'm still wavering if a blog spam list should be part of multi. There are programs that use multi but (unadvisedly) don't differentiate between the source lists. That kind of argues for keeping multi focussed on only mail spam and making a blog spam list separate. On the other hand there's much less overhead in adding a list internally to multi than setting up a whole new list. > It probably would also be good to devote some thought to how entries will be > added to this list and validated. We surely don't want some annoyed blog > spammer spamming the list with every valid doamin they can find! Yes, data quality is always an issue. Any of these ventures will struggle if spammers are able to poison the data. Keeping legitimate domains out of any feed is key and provisions would need to be made for that. Jeff C. -- Jeff Chan mailto:jeffc@surbl.org http://www.surbl.org/

1 0

Re: [SURBL-Discuss] Proposing a greylist
by Rob McEwen 03 Sep '04

03 Sep '04

> OK, here are a few thoughts, after reading this thread, and making a few > specific replies. I think that combining (1) Ryan's specific proposal in his previous message ...with... (2) the cautions in my "just to be sure... so that... unconfirmed.surbl.org will get too convoluted and too bulky to be effective" previous message... (as well as keeping unconfirmed.surbl.org OUT of multi.surbl.org) ...will be a winner! What do you think Jeff?

2 1

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Discuss September 2004