- Discuss - Mailman 3

RE: [SURBL-Discuss] Re: SP91011 your recent report to eBay's Trus t and Safety Department (KMM157050156V37604L0KM)
by Chris Santerre 05 Mar '05

05 Mar '05

> >And their abuse doesnt even respond, so perhaps someone with clue is >reading along ;) > I've had it with this kind of stupidity. Ebay allowing open redirs is simply retarded. I emailed the whole silly story to <thescreensavers(a)g4tv.com> with a quick explanation. Lets see if ebay and Cnet respond when the media gets involved. --Chris

4 3

Redir checking... Was: a too-long thread title
by Matthew Wilson 04 Mar '05

04 Mar '05

> Oh, which [begs] the question: Matthew did you mean on the > application side for checking live messages, or on the data > side, where we build the lists? I was referring to the application side, but I can see how it could also be done on the data side. -Matthew

1 0

RE: [SURBL-Discuss] Re: SP91011 your recent report to eBay's Trus tand Safety Department (KMM157050156V37604L0KM)
by Chris Santerre 04 Mar '05

04 Mar '05

> >Isn't there a way for SURBL to detect these encoded redirect >destination >URLs and check those (without the prepended redirector)? > umm.....sort of. We have some beta research tools that do that, but nothing in the live code. --Chris

2 1

RE: [SURBL-Discuss] Re: SP91011 your recent report to eBay's Trustand Safety Department (KMM157050156V37604L0KM)
by Matthew Wilson 04 Mar '05

04 Mar '05

Isn't there a way for SURBL to detect these encoded redirect destination URLs and check those (without the prepended redirector)?

2 1

Re: SP91011 your recent report to eBay's Trust and Safety Department (KMM157050156V37604L0KM)
by Kevin A. McGrail 04 Mar '05

04 Mar '05

Dear eBay: Wow, your form letter has changed my mind. Your security is perfect. Your commitment to security is stellar. Running an open redirector is a great idea. Sorry I didn't see the light earlier. However, on a new topic, I was shocked and dismayed that eBay is allowing and assumingly SUPPORTING pornography to be distributed through your website. Does this include child pornography or is that only in Europe and places where the age of consent for pornography is under 18? Please advise based on the following link from eBay --WARNING: The following pages contains naked photos: http://cgi4.ebay.com/ws/eBayISAPI.dll?MfcISAPICommand=RedirectToDomain&Doma… What is the meaning of this? eBay is facilitating porn now? OK, now that I have your attention maybe this extreme last resort will ACTUALLY get you to forward this to someone at your company with an understanding of phishing and security that is slightly higher than the Trust and Safety department? If not, I give up and wish you well in your support of the child pornography industry that your company is facilitating by turning a blind eye to glaring security issues. Sincerely, Kevin A. McGrail ----- Original Message ----- From: "eBay Customer Support" <rswebhelp(a)ebay.com> To: "Kevin A. McGrail" <kmcgrail(a)pccc.com> Sent: Saturday, February 26, 2005 12:06 PM Subject: RE: SP91011 your recent report to eBay's Trust and Safety Department (KMM157050156V37604L0KM) > Hello, > > Thank you for writing back. > > I truly apologize if you felt we were not concerned about the email you > received. We are aware of the potential for fraud that these emails > pose. > > Let me assure you that we do work actively and aggressively in > partnership with many agencies, ISP's, and law enforcement groups to > investigate these fraudulent entities. Please keep in mind that eBay is > a public company and not associated with any legislative or police > entity. We rely on the same agencies you do to pursue these fraudulent > activities. We are very much concerned about our member's safety, but we > cannot control the actions of those intent on committing fraud. > > If you have already received a spoofed email once, your email address > has already been harvested. Sadly, you may continue to receive spoofed > emails for some time as these groups migrate from ISP to ISP setting up > fraudulent sites or sending fraudulent emails. > > We advise you to be very cautious of all email messages that ask you to > submit information such as your credit card number or your email > password. eBay (and most other Internet companies) will never ask you > for sensitive personal information such as passwords, bank account or > credit card numbers, Personal Identification Numbers (PINs), or Social > Security numbers in an email. If you ever need to provide information to > eBay please open a new Web browser, type www.ebay.com, and click on the > "site map" link located at the top the page to access the eBay page you > need. > > To keep your eBay experience safe, we have set up a new tutorial about > Spoof Emails to educate our members spotting a fake email. To check it > out, please click on the help link located at the top of all eBay page. > Once the help window appears, click on the link to eBay's Security > Center. From the Security Center you will find a variety of safety > related links. On the right hand side you will see a link to "Protect > yourself from spoof emails". > > Help > Security Center > Protect yourself from spoof emails > > Once again, thank you for alerting us to the spoof email you received. > Your vigilance helps us ensure that eBay remains a safe and vibrant > online marketplace. > > > Regards, > > Marcel > eBay SafeHarbor > Investigations Team > ______________________________ > eBay Inc. > The World's Online Marketplace® > ******************************************* > > Important: eBay will not ask you for sensitive personal information > (such as your password, credit card and bank account numbers, Social > Security numbers, etc.) in an email. Learn more account protection tips > at: > > http://pages.ebay.com/help/confidence/isgw-account-theft-reporting.html > > _____________________________________________ > > For our latest announcements, please check: > > http://www2.ebay.com/aw/announce.shtml > _____________________________________________ > > In order to better serve you, we'd occasionally like to > request feedback on our service. If you would rather > not participate, please click on the link below and send > us an email with the word "REMOVE" in the subject line. > If that does not work, please send an email to the > email address below. Your request will be processed > within 5 days. > > mailto:cssremove@ebay.com > > ******************************************* >

8 22

RE: [SURBL-Discuss] Help for the Windows Guy!
by Jeff Baker 03 Mar '05

03 Mar '05

We use Exchange 2000 on our back-end server which is a Compaq DL380 w/4gb memory. Currently we use McAfee's Groupshield on this unit. I'm not sure on an external call from Ex 2000. What was your experience with spam before and after implementation of SA? -----Original Message----- From: discuss-bounces(a)lists.surbl.org [mailto:discuss-bounces@lists.surbl.org] On Behalf Of Fred Sent: Tuesday, March 01, 2005 5:34 PM To: SURBL Discussion list Subject: Re: [SURBL-Discuss] Help for the Windows Guy! I am one of the first to have jumped on this wagon, what type of mail server are you planning to install this in? Does it provide some way to tie in an external program? Each mail server is very different how they handle this, we used Vopmail and it provides what's called an "agent" which is a batch file you specify to be executed upon arrivial of mail. This works fine but you need to know how to program batch files... How you accomplish this depends on what your mail server allows you to do. It sure is easier setting up a freebsd box to run only e-mail and not have to worry about it for a long time! We haven't touched our mail servers except to upgrade our clamAV once a year. When I ran SA under Win32, I had to keep a close eye on things to make sure they didn't blow up and let me assure you THEY WILL! Frederic Tarasevicius Internet Information Services, Inc. http://www.i-is.com/ 810-794-4400 Jeff Baker wrote: > I want to implement SpamAssassin <http://spamassassin.apache.org/> 3 > but I am just a Windows guy. I use McAfee's Spamkiller 2.7 on their > e500 (linux) and it's just not doing the job. Someone please step me > through the process of where and what to place SpamAssassin on. I see > the wonderful reviews but this Windows guy knows nothing about linux. > > > > Thank anyone very much!!!! > > > > Jeff Baker > > Network Systems Administrator > > The Pantry, Inc. > > Sanford, NC 27330 > > jeff.baker(a)thepantry.com > > > > > > All outbound email messages have been scanned for viruses and content > with the e500 web appliance. > _______________________________________________ > Discuss mailing list > Discuss(a)lists.surbl.org > http://lists.surbl.org/mailman/listinfo/discuss _______________________________________________ Discuss mailing list Discuss(a)lists.surbl.org http://lists.surbl.org/mailman/listinfo/discuss This message has been scanned for viruses and content with the e500 web appliance. All outbound email messages have been scanned for viruses and content with the e500 web appliance.

4 3

Help for the Windows Guy!
by Jeff Baker 02 Mar '05

02 Mar '05

I want to implement SpamAssassin <http://spamassassin.apache.org/> 3 but I am just a Windows guy. I use McAfee's Spamkiller 2.7 on their e500 (linux) and it's just not doing the job. Someone please step me through the process of where and what to place SpamAssassin on. I see the wonderful reviews but this Windows guy knows nothing about linux. Thank anyone very much!!!! Jeff Baker Network Systems Administrator The Pantry, Inc. Sanford, NC 27330 jeff.baker(a)thepantry.com All outbound email messages have been scanned for viruses and content with the e500 web appliance.

5 4

RE: [SURBL-Discuss] Help for the Windows Guy!
by Joe Zitnik 02 Mar '05

02 Mar '05

Jeff, We use a product called Guinevere that has SA integration against GroupWise. The person who developed the app has written a how to on SA on Win32. I cna't promise anything, but maybe the attached link will help. http://www.openhandhome.com/howtosa300.html >>> jeff.baker(a)thepantry.com 03/02 8:42 AM >>> We use Exchange 2000 on our back-end server which is a Compaq DL380 w/4gb memory. Currently we use McAfee's Groupshield on this unit. I'm not sure on an external call from Ex 2000. What was your experience with spam before and after implementation of SA? -----Original Message----- From: discuss-bounces(a)lists.surbl.org [mailto:discuss-bounces@lists.surbl.org] On Behalf Of Fred Sent: Tuesday, March 01, 2005 5:34 PM To: SURBL Discussion list Subject: Re: [SURBL-Discuss] Help for the Windows Guy! I am one of the first to have jumped on this wagon, what type of mail server are you planning to install this in? Does it provide some way to tie in an external program? Each mail server is very different how they handle this, we used Vopmail and it provides what's called an "agent" which is a batch file you specify to be executed upon arrivial of mail. This works fine but you need to know how to program batch files... How you accomplish this depends on what your mail server allows you to do. It sure is easier setting up a freebsd box to run only e-mail and not have to worry about it for a long time! We haven't touched our mail servers except to upgrade our clamAV once a year. When I ran SA under Win32, I had to keep a close eye on things to make sure they didn't blow up and let me assure you THEY WILL! Frederic Tarasevicius Internet Information Services, Inc. http://www.i-is.com/ 810-794-4400 Jeff Baker wrote: > I want to implement SpamAssassin <http://spamassassin.apache.org/> 3 > but I am just a Windows guy. I use McAfee's Spamkiller 2.7 on their > e500 (linux) and it's just not doing the job. Someone please step me > through the process of where and what to place SpamAssassin on. I see > the wonderful reviews but this Windows guy knows nothing about linux. > > > > Thank anyone very much!!!! > > > > Jeff Baker > > Network Systems Administrator > > The Pantry, Inc. > > Sanford, NC 27330 > > jeff.baker(a)thepantry.com > > > > > > All outbound email messages have been scanned for viruses and content > with the e500 web appliance. > _______________________________________________ > Discuss mailing list > Discuss(a)lists.surbl.org > http://lists.surbl.org/mailman/listinfo/discuss _______________________________________________ Discuss mailing list Discuss(a)lists.surbl.org http://lists.surbl.org/mailman/listinfo/discuss This message has been scanned for viruses and content with the e500 web appliance. All outbound email messages have been scanned for viruses and content with the e500 web appliance. _______________________________________________ Discuss mailing list Discuss(a)lists.surbl.org http://lists.surbl.org/mailman/listinfo/discuss

1 0

Phishing reporting addresses
by Jeff Chan 02 Mar '05

02 Mar '05

Here are some addresses to report phishing to: postmaster@corp.mailsecurity.net.au,reportphishing@antiphishing.org, spam@uce.gov,spam@mailpolice.com,scams@fraudwatchinternational.com The mailsecurity and mailpolice ones feed into ph.surbl.org. Hopefully some of the other ones may also eventually. Cheers, Jeff C. -- "If it appears in hams, then don't list it."

1 0

RE: [SURBL-Discuss] Fw: Account Verification
by Chris Santerre 21 Feb '05

21 Feb '05

Let me know and we could forward Kevin's post to the SPAM-L list. That would create some preasure ;) --Chris (Top posting because of my stupid MUA!!!) > Thanks Kevin, > I'm asking around if anyone has contacts at eBay. I've heard back from some folks at eBay that they're now working on this issue. Jeff C.

1 0

linkshield(dot)com ?
by RD 21 Feb '05

21 Feb '05

Hello SURBL Team, Please review linkshield(dot)com. It's a URL cloaking service but no abuse policy, etc. It may have legit uses but their service may actually benefits spammers. Just wanted to get everyone's opinion :-) -RD

3 4

Blind CC's
by Doc Schneider 21 Feb '05

21 Feb '05

I'm asking here because I know some of you probably can figure this one out. My Wife owns raogk.org (Random Acts Of Genealogical Kindness) and has a person who is having e-mail issues. A bit of background. I designed the full backend of this site which once you figure out the country,state,county of where you're wanting information you click on the volunteers name and a form pops up which allows you to enter your name/e-mail and what you want this person to lookup. And once it is sent it sends you a BCC of what you asked. Anyway, this one person is using MSN. And is not getting his BCC's we've had him do a complete search through his Outlook Express for the subject of these. Nada. And we've had no complaints from anyone else using MSN, so doubt it is the issue. Could this person be running something like an e-mail filter which is eatting his BCCs? Any of you heard of this? We're not sure what is running on this persons computer, so unknown if or what could be causing this. I'm out of ideas. We have the headers from some test messages we had this person do. Funny thing is they're getting all the other @raogk.org e-mail. Please include the raogk-admin(a)raogk.org address if you reply. Thanks, -Doc

2 1

Fw: Account Verification
by Kevin A. McGrail 21 Feb '05

21 Feb '05

This is a follow-up to my initial discovery that eBay has it's own redirector and this redirector was now showing up in Phishing scams. Despite my adamant, fervent & rabid inquiries, eBay has done nothing. With the rise of the use of the redirector on eBay and this more obscure url now being used, I believe even more phish-aware users would be caught: http://cgi4-munged.ebay.com/ws/eBayISAPI.dll?MfcISAPICommand=RedirectToDoma… Anyone who knows anyone at eBay that understands security should email them and tell them to turn this redirector OFF. In the meantime, here's an SA Rule to help catch it which I would appreciate feedback about: # This rule is to mark emails using the exploit of the eBay redirector uri KAM_EBAYREDIR /.*.ebay.com.*RedirectToDomain/i describe KAM_EBAYREDIR Attempted use of eBay redirector - high probability of fraud score KAM_EBAYREDIR 7.0 More posted at: http://www.peregrinehw.com/downloads/SpamAssassin/contrib/KAM.cf Regards, KAM

2 2

RE: [SURBL-Discuss] Is something wrong with SURBL-checker?
by Chris Santerre 18 Feb '05

18 Feb '05

> >Jeff Chan wrote: > >> I'm not getting matches either. Let's ask Dallas to please look >> into it for us. > >Alright! I noticed spam got hits from SURBL-lists anyway (as >you pointed >out in your other message). > >I'll wait report my spam-mails until the problems are solved. > I sent the Ninja in charge an email. We recently did an update that may have messed up the public one. --Chris

2 1

RE: [SURBL-Discuss] Fw: TKO Notice: Urgent Fraud Investigation
by Matthew.van.Eerde＠hbinc.com 17 Feb '05

17 Feb '05

Rob McEwen wrote: >> http://cgi4.ebay.com/ws/eBayISAPI.dll?MfcISAPICommand=RedirectToDomain&Doma >> inUrl=http://mymt.co.kr/.cgi-bin/eBaySuspension/signin.ebay.com/aw->cgi/sec > ure/eBayISAPI.dllSignIn-ssPageName->hhsin.php?MfcISAPICommand=SignInFPP&Usin > gSSL=1&email= > >> Erm, it's called a redirector. Did you try the >> URL? ebay's site redirects >> to the URL in the DomainURL parameter. > > Whatever you call it, it's bad news for any parser which might not > grab and extract the referenced URL for SURBL checking. > > Also, this leads to additional questions: > > (1) Are there legitimate "business purposes" for ebay to have such a > redirector in the first place? To a certain limited extent, yes > (2) If so, are there legitimate reasons for such a redirector to EVER > show up in legitimate e-mails? To that extent, yes > (3) If not, does anyone know of a "clearinghouse" page where ALL such > types of redirectors are listed so that rules could be built to block > e-mails containing these (using rules-based blocking)? Also, are > there already SA rules for such? > > Rob McEwen eBay should certainly realize that they are imparting a degree of authority to URLs that are redirected in this manner. They may even be liable for damages. Best practices probably dictate that they keep a list of URLs that are legitimate redirection destinations, and limit redirection to those URLs - on attempts to feed the redirector any other URL, they should pop up big ugly error messages saying "someone's trying to phish you (or maybe we forgot to update our list)" Matthew.van.Eerde (at) hbinc.com 805.964.4554 x902 Hispanic Business Inc./HireDiversity.com Software Engineer perl -e"map{y/a-z/l-za-k/;print}shift" "Jjhi pcdiwtg Ptga wprztg,"

1 0

Is something wrong with SURBL-checker?
by Martin 15 Feb '05

15 Feb '05

Hi, Is there something wrong with the SURBL-checker at http://www.rulesemporium.com/cgi-bin/uribl.cgi ? I get no matches of any domains i'm testing. Even with spam-domains that hits the multi-list isn't listed as blocked. Is this problem related to the "FP-rate" thread? / Martin

2 3

RE: [SURBL-Discuss] FP rate?
by Chris Santerre 14 Feb '05

14 Feb '05

>-----Original Message----- >From: Fred [mailto:tech2@i-is.com] >Sent: Monday, February 14, 2005 1:26 PM >To: SURBL Discussion list >Subject: Re: [SURBL-Discuss] FP rate? > > >Chris Santerre wrote: >> Can we trust the FP rate with the current bug in SA? > >Not taking sides but it might be a bug in Net::DNS, the SA >devs have not >exactly tied down what was causing this issue. There was talk >of re-write >in the way they use Net::DNS to possibly fix this issue but >I'm pretty sure >this was not SA specific. > >http://bugzilla.spamassassin.org/show_bug.cgi?id=3997 > Oh I agree. I don't know what is causing it, but I know it must be throwing off the reported FP rate. Although proably for all the URIRBLs. I'd love to get a monthly report from DQ on his rates. But I know he is busy. --Chris

4 3

RE: [SURBL-Discuss] FP rate?
by Chris Santerre 14 Feb '05

14 Feb '05

Can we trust the FP rate with the current bug in SA? -_Chris

2 1

Re: [SURBL-Discuss] ANN: new surbl client (still beta)
by Jeff Chan 14 Feb '05

14 Feb '05

On Saturday, February 12, 2005, 3:36:11 AM, Alain Alain wrote: > Hi Jeff >> On Saturday, February 12, 2005, 2:34:20 AM, Alain Alain wrote: >> >> Generally speaking it may be better to apply this kind of >> >> filtering at the server level since there are economies of scale, >> >> especially in terms of things like DNS lookups and caching. If >> >> we suddenly get 100k more DNS clients, that could tax the name >> >> servers somewhat. If those same 100k users were using 100 >> >> servers instead, the DNS loading would be quite a bit less. In >> >> that sense centralization is desirable. >> >> > Mmmm isn't the dns server from the ISP caching the dns requests? I >> > would think it doesn't make a big difference (except when a server is >> > rsync'ing). The difference could be that end users check their e-mail >> > not when arriving on the MTA, but later. >> >> One difference is that the ISP's mail server may see many of the >> same spams within a short period of time, and the lookups would >> probably tend to be cached over that time span. Individual users >> may POP or IMAP their messages at any random time, so the DNS >> cache hit rate may be lower for them. > This will only the case for spam e-mail, not for domains inside ham e-mail. But most well-written applications, e.g. SpamAssassin, are already ignoring most ham URIs due to local whitelisting, so it's spam URI domain caching that's the main issue. >> I think we're agreeing, but I've never tried to quantify the >> difference between these. We can propose that there's some >> difference but how much is unknown. I would suggest a pretty >> strong cache effect for mail servers however. > But the good news is : The more users, the more caching. So the > burden on the nameservers will grow slower. The SURBL zone files have a minimal 15 minute TTL, so in order for ISP resolver hits to be cached, the queries will need to occur within some 15 minutes, which seems less likely at MUA download time than at MTA processing time. MTAs probably see similar spam over a short period of time whereas MUA clients can download at any later time. In this case, I don't think your argument applies. For something like caching yahoo domains, or any with "normal" longer TTLs, it probably applies more strongly. Jeff C. -- "If it appears in hams, then don't list it."

2 2

Re: [SURBL-Discuss] FP rate?
by Alain 13 Feb '05

13 Feb '05

Hi Jeff > >> > I know that not all FP's are reported and there are > >> > probably no exact numbers, but it should give a good idea. Or am I > >> > wrong? > >> > >> The FP reports are probably too few overall to be meaningful in > >> terms of differentiating performance between lists. There just > >> aren't that many, maybe a few a day on average. > >> > > > Yes, but I wasn't thinking on differentiating between the lists, there > > are other results for. What I was thinking on was the number of FP's > > that exists on more than one list. This is very usefull information > > when combining lists. If almost no FP's do occur on more than one > > list (at the same time) requiring appearance on at least 2 lists > > would be a very safe one. > > Good point. Anecdotally, FPs don't tend to appear on multiple > lists very often, at least the FPs we've seen reported. This is > unmeasured, just a subjective opinion. If we had some of the > list data in combined form as I had proposed then we could test > it better. I suppose I could just do it. ;-) > I f the reported one's are very rare, this would probably even more the case for the not reported one's. If there's a FP the chance for being reported will grow if on more than one list. Mmm the combined lists just have to be available to someone with a big ham corpus, to test it. Personaly knowing the results for "at least 2" or "at least 3" , would be nice. It also would be nice to know how those combination would result inside : http://www.surbl.org/permuted-hits.out.txt Alain

3 5

Re: [SURBL-Discuss] ANN: new surbl client (still beta)
by Jeff Chan 13 Feb '05

13 Feb '05

On Saturday, February 12, 2005, 2:41:36 AM, Alain Alain wrote: >> >> - I've added a local skiplist with about top half of the public >> >> "whitelist", no need to query those. >> >> When you say half, that may be more than optimal (should be about >> 5000 records). SpamAssassin is using the top 125, which worked >> out to about the 50%th percentile of all whitelist hits when we >> first set this up. (Now that result is skewed *because* >> SpamAssassin isn't checking those 125 any more, but their >> snapshot of the 125 is still probably useful. >> >> I'd say anything between 100 and 1000 would probably be a good >> compromise between list size and coverage. > The only disadvantage I see from a bigger local skiplist is some local > CPU usage for every uri in a email. Most pc's have plenty of CPU > power ;-) If this could become a problem, I can lower or optimise the > local checking. Are there any other disadvantages? One reason SpamAssassin didn't want to hard code too many domains into their local whitelist was in case we needed to withdraw any, i.e. because they started spamming. The time between code releases can be many months, and some people may never update, so they wanted to be sure to get very hammy domains into that list. (While Yahoo and Microsoft probably aren't going to start spamming any time soon, that may be less certain about some of the less commonly seen domains.) But I'm glad that you're trying to minimize the DNS queries. Jeff C. -- "If it appears in hams, then don't list it."

2 2

Re: [SURBL-Discuss] FP rate?
by Jeff Chan 12 Feb '05

12 Feb '05

On Saturday, February 12, 2005, 3:09:46 AM, Alain Alain wrote: >> > I know that not all FP's are reported and there are >> > probably no exact numbers, but it should give a good idea. Or am I >> > wrong? >> >> The FP reports are probably too few overall to be meaningful in >> terms of differentiating performance between lists. There just >> aren't that many, maybe a few a day on average. >> > Yes, but I wasn't thinking on differentiating between the lists, there > are other results for. What I was thinking on was the number of FP's > that exists on more than one list. This is very usefull information > when combining lists. If almost no FP's do occur on more than one > list (at the same time) requiring appearance on at least 2 lists > would be a very safe one. Good point. Anecdotally, FPs don't tend to appear on multiple lists very often, at least the FPs we've seen reported. This is unmeasured, just a subjective opinion. If we had some of the list data in combined form as I had proposed then we could test it better. I suppose I could just do it. ;-) Jeff C. -- "If it appears in hams, then don't list it."

1 0

Re: [SURBL-Discuss] FP rate?
by Jeff Chan 12 Feb '05

12 Feb '05

On Friday, February 11, 2005, 5:29:29 PM, Alain Alain wrote: >> That said, here are some results Daniel Quinlan posted from the >> mass-checks on the SpamAssassin corpora around 26 January 2005: >> >> > Weekly mass-check results for SURBL: >> >> >OVERALL% SPAM% HAM% S/O RANK SCORE NAME >> > 217996 164295 53701 0.754 0.00 0.00 (all messages) >> >100.000 75.3661 24.6339 0.754 0.00 0.00 (all messages as %) >> > 11.644 15.4490 0.0037 1.000 0.98 3.90 URIBL_SC_SURBL >> > 39.572 52.4976 0.0261 1.000 0.98 3.00 URIBL_JP_SURBL >> > 51.955 68.9236 0.0391 0.999 0.96 2.00 URIBL_OB_SURBL >> > 5.690 7.5492 0.0000 1.000 0.95 2.01 URIBL_AB_SURBL >> > 53.948 71.5238 0.1769 0.998 0.83 0.54 URIBL_WS_SURBL >> > 0.030 0.0396 0.0000 1.000 0.51 0.84 URIBL_PH_SURBL >> > Am I right with the following : > JP has 0.0261% FP on 24.6339% of all msg --> 0.0065% of all msg > (is less than 1 in 15.000) That sounds right, but the particular proportions of spam versus ham may not be meaningful, i.e. they may not be representative of an actual mail stream. So the percentages are probably more usefully compared only to spam or ham and not to a combined total of messages. Certainly the relative percentages within spam or ham are meaningful and mostly useful with the caveat that the spam detection rates are wrong for quickly moving data in SC and AB since the test corpora cover too much time for them. (This is more true for spam than ham since spam domains vary quickly with time, but ham domains are relatively steady.) >> SC and AB have much better real world results than show above >> because their time period is much shorter than the test >> corpora's. > Yes, but maybe the FP's will grow faster ;-) That tends not to be the case. The SpamCop data is filtered multiple times and is human-checked at the front end. The SC FP rates are consistently among the lowest, and the spam detection rates are very high for a very small list. In short it's an effective strategy. >> Also note that the JP data is now removed from the WS data, and >> some old data was removed from WS. So the WS spam and ham hit >> rates have probably both decreased since this check was done. >> JP should be about the same. > That will show in the future. Is also a good thing. Yes, it's fairer to the data sources. >> > And if possible, has anybody statistics from FP's that where on >> > several of the sublists -at the same time-? > [snip] >> I don't think that is known yet. I had proposed setting up some >> test lists with combinations like this, but got no response. ;-) >> >> If it *is* known I think we'd all like to hear about it. :-) > I think it could be known to the great people that check the FP > reports. Normally they check against all sublists (I hope) and fix > them all. When we whitelist a domain, it's excluded from all SURBLs. The original data source is usually notified. > I know that not all FP's are reported and there are > probably no exact numbers, but it should give a good idea. Or am I > wrong? The FP reports are probably too few overall to be meaningful in terms of differentiating performance between lists. There just aren't that many, maybe a few a day on average. Jeff C. -- "If it appears in hams, then don't list it."

1 0

Re: [SURBL-Discuss] ANN: new surbl client (still beta)
by Alain 12 Feb '05

12 Feb '05

Hi Jeff > On Saturday, February 12, 2005, 2:34:20 AM, Alain Alain wrote: > >> Generally speaking it may be better to apply this kind of > >> filtering at the server level since there are economies of scale, > >> especially in terms of things like DNS lookups and caching. If > >> we suddenly get 100k more DNS clients, that could tax the name > >> servers somewhat. If those same 100k users were using 100 > >> servers instead, the DNS loading would be quite a bit less. In > >> that sense centralization is desirable. > > > Mmmm isn't the dns server from the ISP caching the dns requests? I > > would think it doesn't make a big difference (except when a server is > > rsync'ing). The difference could be that end users check their e-mail > > not when arriving on the MTA, but later. > > One difference is that the ISP's mail server may see many of the > same spams within a short period of time, and the lookups would > probably tend to be cached over that time span. Individual users > may POP or IMAP their messages at any random time, so the DNS > cache hit rate may be lower for them. This will only the case for spam e-mail, not for domains inside ham e-mail. > > I think we're agreeing, but I've never tried to quantify the > difference between these. We can propose that there's some > difference but how much is unknown. I would suggest a pretty > strong cache effect for mail servers however. But the good news is : The more users, the more caching. So the burden on the nameservers will grow slower. Alain

1 0

Re: [SURBL-Discuss] FP rate?
by Alain 12 Feb '05

12 Feb '05

Hi Jeff > >> That said, here are some results Daniel Quinlan posted from the > >> mass-checks on the SpamAssassin corpora around 26 January 2005: > >> > >> > Weekly mass-check results for SURBL: > >> > >> >OVERALL% SPAM% HAM% S/O RANK SCORE NAME > >> > 217996 164295 53701 0.754 0.00 0.00 (all messages) > >> >100.000 75.3661 24.6339 0.754 0.00 0.00 (all messages as %) > >> > 11.644 15.4490 0.0037 1.000 0.98 3.90 URIBL_SC_SURBL > >> > 39.572 52.4976 0.0261 1.000 0.98 3.00 URIBL_JP_SURBL > >> > 51.955 68.9236 0.0391 0.999 0.96 2.00 URIBL_OB_SURBL > >> > 5.690 7.5492 0.0000 1.000 0.95 2.01 URIBL_AB_SURBL > >> > 53.948 71.5238 0.1769 0.998 0.83 0.54 URIBL_WS_SURBL > >> > 0.030 0.0396 0.0000 1.000 0.51 0.84 URIBL_PH_SURBL > >> > > > Am I right with the following : > > > JP has 0.0261% FP on 24.6339% of all msg --> 0.0065% of all msg > > (is less than 1 in 15.000) > > That sounds right, but the particular proportions of spam versus > ham may not be meaningful, i.e. they may not be representative > of an actual mail stream. So the percentages are probably more > usefully compared only to spam or ham and not to a combined total > of messages. ok > > Certainly the relative percentages within spam or ham are > meaningful and mostly useful with the caveat that the spam > detection rates are wrong for quickly moving data in SC and AB > since the test corpora cover too much time for them. (This is > more true for spam than ham since spam domains vary quickly with > time, but ham domains are relatively steady.) > ok > >> SC and AB have much better real world results than show above > >> because their time period is much shorter than the test > >> corpora's. > > > Yes, but maybe the FP's will grow faster ;-) > > That tends not to be the case. The SpamCop data is filtered > multiple times and is human-checked at the front end. The SC FP > rates are consistently among the lowest, and the spam detection > rates are very high for a very small list. In short it's an > effective strategy. > ok and I am overall impressed with the low FP rates on all lists. > >> Also note that the JP data is now removed from the WS data, and > >> some old data was removed from WS. So the WS spam and ham hit > >> rates have probably both decreased since this check was done. > >> JP should be about the same. > > > That will show in the future. Is also a good thing. > > Yes, it's fairer to the data sources. > > >> > And if possible, has anybody statistics from FP's that where on > >> > several of the sublists -at the same time-? > > > [snip] > > >> I don't think that is known yet. I had proposed setting up some > >> test lists with combinations like this, but got no response. ;-) > >> > >> If it *is* known I think we'd all like to hear about it. :-) > > > I think it could be known to the great people that check the FP > > reports. Normally they check against all sublists (I hope) and fix > > them all. > > When we whitelist a domain, it's excluded from all SURBLs. The > original data source is usually notified. > > > I know that not all FP's are reported and there are > > probably no exact numbers, but it should give a good idea. Or am I > > wrong? > > The FP reports are probably too few overall to be meaningful in > terms of differentiating performance between lists. There just > aren't that many, maybe a few a day on average. > Yes, but I wasn't thinking on differentiating between the lists, there are other results for. What I was thinking on was the number of FP's that exists on more than one list. This is very usefull information when combining lists. If almost no FP's do occur on more than one list (at the same time) requiring appearance on at least 2 lists would be a very safe one. Alain

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Discuss