David Hooton has made available to us MailSecurity's spamvertised site data for public use. I have turned that data into a SURBL list for testing:
ms.surbl.org
This is the same company that has provided the phishing data currently in multi.surbl.org.
Please do corpus checks, etc. but don't use it on high volume mail servers yet since it's only hosted on my mail server right now. Please let us know what spam detection rates you get and particularly about false positive rates.
At this point we don't know if the data will stay a separate list, be added to multi only, get folded into ws, etc.
The results you report will help us decide that.
Jeff C.
On Thu, 29 Jul 2004 02:35:50 -0700, Jeff Chan jeffc@surbl.org wrote:
David Hooton has made available to us MailSecurity's spamvertised site data for public use. I have turned that data into a SURBL list for testing:
ms.surbl.org
This is the same company that has provided the phishing data currently in multi.surbl.org.
Please do corpus checks, etc. but don't use it on high volume mail servers yet since it's only hosted on my mail server right now. Please let us know what spam detection rates you get and particularly about false positive rates.
OK... just installed it in my own (low trafic) server... will let you know how it goes...
At this point we don't know if the data will stay a separate list, be added to multi only, get folded into ws, etc.
FWIW... it's nice to have the data separate (since I still can't upgrade to SA3)... I also think that ph could be made a separate list no matter it is small, I think it's not much overhead to have one subdomain more or less... or is it?
Regards, and thanx for all you do.
On Thursday, July 29, 2004, 5:53:48 AM, Mariano Absatz wrote:
On Thu, 29 Jul 2004 02:35:50 -0700, Jeff Chan jeffc@surbl.org wrote:
At this point we don't know if the data will stay a separate list, be added to multi only, get folded into ws, etc.
FWIW... it's nice to have the data separate (since I still can't upgrade to SA3)... I also think that ph could be made a separate list no matter it is small, I think it's not much overhead to have one subdomain more or less... or is it?
The overhead in setting up a new list is mainly in coordinating with the folks providing public DNS and also of course letting SURBL users know that there's a new list. And there's a little other administration to do for each one. But once they're set up, they pretty much run without much intervention.
Based on early feedback this new data may be best suited for inclusion in ws, but it would help to get more feedback from folks about what results they get.
Regards, and thanx for all you do.
Thanks, but it's not just me. There are lots of people helping out, including all the people providing feedback here, such as yourself. :-)
Our thanks to everyone!
Jeff C.
Jeff Chan wrote:
David Hooton has made available to us MailSecurity's spamvertised site data for public use.
Thanks, David.
I have turned that data into a SURBL list for testing:
ms.surbl.org
I'm using the following settings in uribl.cf:
urirhsbl URIBL_MS_SURBL ms.surbl.org. A header URIBL_MS_SURBL eval:check_uridnsbl('URIBL_MS_SURBL') describe URIBL_MS_SURBL URL listed in the MS SURBL blocklist tflags URIBL_MS_SURBL net score URIBL_MS_SURBL 1.0
Should the Link http://test.surbl.org.ms.surbl.org/foo.html in a test message cause SpamAssassin 3.0 to report a URIBL_MS_SURBL hit?
On Thursday, July 29, 2004, 8:40:26 AM, Ralph Seichter wrote:
I'm using the following settings in uribl.cf:
urirhsbl URIBL_MS_SURBL ms.surbl.org. A header URIBL_MS_SURBL eval:check_uridnsbl('URIBL_MS_SURBL') describe URIBL_MS_SURBL URL listed in the MS SURBL blocklist tflags URIBL_MS_SURBL net score URIBL_MS_SURBL 1.0
Should the Link http://test.surbl.org.ms.surbl.org/foo.html in a test message cause SpamAssassin 3.0 to report a URIBL_MS_SURBL hit?
A test URI would look like this:
http://surbl-org-permanent-test-point-MUNGED.com/
(without -MUNGED) and it should hit on all SURBLs.
http://www.surbl.org/faq.html#test-uris
We had to come up with that bogus two level domain since the code that does SURBL lookups folds .com, etc into two levels before querying.
http://www.surbl.org/faq.html#testpoints
Jeff C.
Jeff Chan wrote:
Should the Link http://test.surbl.org.ms.surbl.org/foo.html in a test message cause SpamAssassin 3.0 to report a URIBL_MS_SURBL hit?
A test URI would look like this: http://surbl-org-permanent-test-point-MUNGED.com/ (without -MUNGED) and it should hit on all SURBLs.
Yes, I read this in the FAQ. My question remains, though, because I'd like to know if test.surbl.org.ms.surbl.org, which returns a hit with dig as you pointed out to me earlier, could also trigger a positive response when used in a URL. I'm looking for URLs which I can use to selectively trigger only one SURBL and not all of them. Sorry for not clarifying this in my original posting.
On Thursday, July 29, 2004, 2:49:46 PM, Ralph Seichter wrote:
I'd like to know if test.surbl.org.ms.surbl.org, which returns a hit with dig as you pointed out to me earlier, could also trigger a positive response when used in a URL.
The manual DNS lookup is a way to check the contents of a list DNS zone, but is not usable in a test URI.
I'm looking for URLs which I can use to selectively trigger only one SURBL and not all of them.
All of the lists have the same (two level domain or IP address) testpoints, so perhaps the only way to test a given list is to find an entry that is unique to only that list and test on it. However the data are fairly dynamic and an entry on one list may get added to another one if the different data sources pick it up.
Another way to test would be to enable using only one list at a time through the config files.
Jeff C.
Jeff Chan wrote:
I'm looking for URLs which I can use to selectively trigger only one SURBL and not all of them.
All of the lists have the same (two level domain or IP address) testpoints
In theory you could add individual 127.0.0.? test points for all lists belonging to "multi" with one exception:
4.0.0.127.ws.surbl.org 8.0.0.127.ph.surbl.org 16.0.0.127.ob.surbl.org 32.0.0.127.ab.surbl.org
2.0.0.127.sc.surbl.org is the exception, 127.0.0.2 is listed everywhere. You could even add the combinations to corresponding lists (e.g. 60 to ws + ph + ob + ab, but not sc, 62 to all lists, 40 only to ab + ph, etc.)
Bye, Frank
On Thursday, July 29, 2004, 4:17:48 PM, Frank Ellermann wrote:
In theory you could add individual 127.0.0.? test points for all lists belonging to "multi" with one exception:
4.0.0.127.ws.surbl.org 8.0.0.127.ph.surbl.org 16.0.0.127.ob.surbl.org 32.0.0.127.ab.surbl.org
2.0.0.127.sc.surbl.org is the exception, 127.0.0.2 is listed everywhere. You could even add the combinations to corresponding lists (e.g. 60 to ws + ph + ob + ab, but not sc, 62 to all lists, 40 only to ab + ph, etc.)
Yes, and Justin Mason requested that the test points in multi return the sum of all constituent lists, i.e. 2 + 4 + 8 + 16 + 32 = 62.
But currently all testpoints resolve to 127.0.0.2, even in multi. There are good reasons for both approaches.
Jeff C.
Jeff Chan wrote:
currently all testpoints resolve to 127.0.0.2, even in multi. There are good reasons for both approaches.
One good reason to avoid 32 (now) resp. 128 (later) test point combinations is that it's more confusing than really useful.
But maybe you could add _one_ multi test point 127.0.0.222 with "host" 222.0.0.127.multi.surbl.org = 127.0.0.222, that should be good enough to test "multi" setups.
Or in other words you could add http://127.0.0.222/ as test point URL to all individual zones except from ab.surbl.org
Bye, Frank
On Thursday, July 29, 2004, 7:24:43 PM, Frank Ellermann wrote:
Jeff Chan wrote:
currently all testpoints resolve to 127.0.0.2, even in multi.
But maybe you could add _one_ multi test point 127.0.0.222 with "host" 222.0.0.127.multi.surbl.org = 127.0.0.222, that should be good enough to test "multi" setups.
Or in other words you could add http://127.0.0.222/ as test point URL to all individual zones except from ab.surbl.org
It's an interesting proposal, but I think we'll not reserve any particular resolved test IPs, because one alternative is to have every bit set in the test answer for every list in multi, and in that case we'd want to let the result simply be the sum of bits for whatever lists happened to exist at the time. If we try to reserve numbers, there could be conflicts with those (or future) bits.
Jeff C.
Pretty vanilla RH 8 machine with Perl 5.8. Any ideas?
Thanks in advance..
CPAN.pm: Going to build E/EA/EAK/Mail-SpamAssassin-SpamCopURI-0.19.tar.gz
Can't locate URI/QueryParam.pm in @INC (@INC contains: lib /usr/lib/perl5/5.8.0/i386-linux-thread-multi /usr/lib/perl5/5.8.0 /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.0 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.0 /usr/lib/perl5/vendor_perl .) at lib/Mail/SpamAssassin/SpamCopURI.pm line 5. BEGIN failed--compilation aborted at lib/Mail/SpamAssassin/SpamCopURI.pm line 5. Compilation failed in require at /usr/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin/PerMsgStatus.pm line 41. BEGIN failed--compilation aborted at /usr/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin/PerMsgStatus.pm line 41. Compilation failed in require at /usr/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin.pm line 62. BEGIN failed--compilation aborted at /usr/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin.pm line 62. Compilation failed in require at Makefile.PL line 3. BEGIN failed--compilation aborted at Makefile.PL line 3. Running make test Make had some problems, maybe interrupted? Won't test Running make install Make had some problems, maybe interrupted? Won't install
Ray Dzek Network Operations Supervisor Specialized Bicycle Components
Install URI from CPAN.
--eric
On Thu, Jul 29, 2004 at 03:22:59PM -0700, Ray Dzek wrote:
Pretty vanilla RH 8 machine with Perl 5.8. Any ideas?
Thanks in advance..
CPAN.pm: Going to build E/EA/EAK/Mail-SpamAssassin-SpamCopURI-0.19.tar.gz
Can't locate URI/QueryParam.pm in @INC (@INC contains: lib /usr/lib/perl5/5.8.0/i386-linux-thread-multi /usr/lib/perl5/5.8.0 /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.0 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.0 /usr/lib/perl5/vendor_perl .) at lib/Mail/SpamAssassin/SpamCopURI.pm line 5. BEGIN failed--compilation aborted at lib/Mail/SpamAssassin/SpamCopURI.pm line 5. Compilation failed in require at /usr/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin/PerMsgStatus.pm line 41. BEGIN failed--compilation aborted at /usr/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin/PerMsgStatus.pm line 41. Compilation failed in require at /usr/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin.pm line 62. BEGIN failed--compilation aborted at /usr/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin.pm line 62. Compilation failed in require at Makefile.PL line 3. BEGIN failed--compilation aborted at Makefile.PL line 3. Running make test Make had some problems, maybe interrupted? Won't test Running make install Make had some problems, maybe interrupted? Won't install
Ray Dzek Network Operations Supervisor Specialized Bicycle Components
Discuss mailing list Discuss@lists.surbl.org http://lists.surbl.org/mailman/listinfo/discuss
On Thursday, July 29, 2004, 3:22:59 PM, Ray Dzek wrote:
Pretty vanilla RH 8 machine with Perl 5.8. Any ideas?
Do you have URI installed? The first few lines of SpamCopURI.pm are:
package Mail::SpamAssassin::SpamCopURI;
use strict; use URI; use URI::QueryParam; use URI::Escape qw(uri_unescape);
Jeff C.
I just got done dealing with a bunch of install errors. I eventually installed everything through CPAN, which includes Mail::SpamAssassin, LWP, URI, Mail::SpamAssassin::SpamCopURI, and Perl. Once I had reinstalled everything in the right order (don't ask what that is cause I don't remember), everything worked like a charm. -David
-----Original Message----- From: discuss-bounces@lists.surbl.org [mailto:discuss-bounces@lists.surbl.org]On Behalf Of Ray Dzek Sent: Thursday, July 29, 2004 3:23 PM To: 'SURBL Discussion list' Subject: [SURBL-Discuss] Install Errors...
Pretty vanilla RH 8 machine with Perl 5.8. Any ideas?
Thanks in advance..
CPAN.pm: Going to build E/EA/EAK/Mail-SpamAssassin-SpamCopURI-0.19.tar.gz
Can't locate URI/QueryParam.pm in @INC (@INC contains: lib /usr/lib/perl5/5.8.0/i386-linux-thread-multi /usr/lib/perl5/5.8.0 /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.0 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.0 /usr/lib/perl5/vendor_perl .) at lib/Mail/SpamAssassin/SpamCopURI.pm line 5. BEGIN failed--compilation aborted at lib/Mail/SpamAssassin/SpamCopURI.pm line 5. Compilation failed in require at /usr/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin/PerMsgStatus.pm line 41. BEGIN failed--compilation aborted at /usr/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin/PerMsgStatus.pm line 41. Compilation failed in require at /usr/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin.pm line 62. BEGIN failed--compilation aborted at /usr/lib/perl5/site_perl/5.8.0/Mail/SpamAssassin.pm line 62. Compilation failed in require at Makefile.PL line 3. BEGIN failed--compilation aborted at Makefile.PL line 3. Running make test Make had some problems, maybe interrupted? Won't test Running make install Make had some problems, maybe interrupted? Won't install
Ray Dzek Network Operations Supervisor Specialized Bicycle Components
_______________________________________________ Discuss mailing list Discuss@lists.surbl.org http://lists.surbl.org/mailman/listinfo/discuss
Hi!
David Hooton has made available to us MailSecurity's spamvertised site data for public use. I have turned that data into a SURBL list for testing:
ms.surbl.org
This is the same company that has provided the phishing data currently in multi.surbl.org.
I have been testing this for 3 hours now, on a large cluster, and so far very few hits on those ones. And the ones that give hits are also listed in WS.
Out of 2200 domains in the list around 800 are 'new' ones, but it seems they are either dead or old :) (read, not used in spam anymore?)
As a seperate list i would so far say, uhm, not really worth the efford, as a extra list of domains for WS, i think its worth it.
Out of 12.000 ones hit by WS i had 2 hits that were only seen by the MS list. I dont think that would be enough. Where is the date taken from? Actual recent spam mails? If so, sounds like a little outdated list ?
Bye, Raymond.
On Thursday, July 29, 2004, 11:11:40 AM, Raymond Dijkxhoorn wrote:
I have been testing this for 3 hours now, on a large cluster, and so far very few hits on those ones. And the ones that give hits are also listed in WS.
Out of 2200 domains in the list around 800 are 'new' ones, but it seems they are either dead or old :) (read, not used in spam anymore?)
As a seperate list i would so far say, uhm, not really worth the efford, as a extra list of domains for WS, i think its worth it.
Out of 12.000 ones hit by WS i had 2 hits that were only seen by the MS list. I dont think that would be enough. Where is the date taken from? Actual recent spam mails? If so, sounds like a little outdated list ?
Sounds like it needs to be pruned for old entries by David's group. Also sounds like it would best be added to ws perhaps.
Would like to hear other people's experiences also.
Cheers,
Jeff C.
On Thursday, July 29, 2004, 11:11:40 AM, Raymond Dijkxhoorn wrote:
Out of 2200 domains in the list around 800 are 'new' ones, but it seems they are either dead or old :) (read, not used in spam anymore?)
Yes, too I found that 1424 of the 2200 or so entries from MailSecurity's data are also on ws.surbl.org. That sounds to me like the new ones should get added to ws.
But if some of those "new" ones are dead or old, we should ask David to prune them from the data first.
I'll probably remove ms.surbl.org in a few days, hoping people will do some more testing.
By the way, for anyone using rbldnsd and rsync, feel free to grab ms.surbl.org.rbldnsd from the rsync server and test the list on your production mail servers. If you run a local copy of the zone file, DNS is not an issue.
Jeff C.
On Thursday, July 29, 2004, 6:12:52 PM, Jeff Chan wrote:
On Thursday, July 29, 2004, 11:11:40 AM, Raymond Dijkxhoorn wrote:
Out of 2200 domains in the list around 800 are 'new' ones, but it seems they are either dead or old :) (read, not used in spam anymore?)
Yes, too I found that 1424 of the 2200 or so entries from MailSecurity's data are also on ws.surbl.org. That sounds to me like the new ones should get added to ws.
But if some of those "new" ones are dead or old, we should ask David to prune them from the data first.
I'll probably remove ms.surbl.org in a few days, hoping people will do some more testing.
Since Raymond is now folding the MailSecurity data into ws.surbl.org, and the overlap with WS is large, I'm going to go ahead and remove the ms.surbl.org list in a few days unless there are any major objections.
Jeff C.
On Sunday, August 1, 2004, 4:02:31 PM, Jeff Chan wrote:
Since Raymond is now folding the MailSecurity data into ws.surbl.org, and the overlap with WS is large, I'm going to go ahead and remove the ms.surbl.org list in a few days unless there are any major objections.
I'll go ahead and ask if anyone wants to keep ms.surbl.org around any longer as a separate list, for example for testing. The MailSecurity data is all in ws.surbl.org now....
If I don't hear any compelling reasons to keep it for a few more days, then we will go ahead and shut it down.
Jeff C.
On Tuesday, August 3, 2004, 2:15:40 AM, Jeff Chan wrote:
On Sunday, August 1, 2004, 4:02:31 PM, Jeff Chan wrote:
Since Raymond is now folding the MailSecurity data into ws.surbl.org, and the overlap with WS is large, I'm going to go ahead and remove the ms.surbl.org list in a few days unless there are any major objections.
I'll go ahead and ask if anyone wants to keep ms.surbl.org around any longer as a separate list, for example for testing. The MailSecurity data is all in ws.surbl.org now....
If I don't hear any compelling reasons to keep it for a few more days, then we will go ahead and shut it down.
OK I've gotten rid of the MS beta list since that data is now in WS.
Raymond, please remove MS zone files from the rsync server.
Anyone who was secondarying, rsyncing or serving it, please stop, etc.
Cheers,
Jeff C.
On Thu, 29 Jul 2004 20:11:40 +0200 (CEST), Raymond Dijkxhoorn raymond@prolocation.net wrote:
Out of 2200 domains in the list around 800 are 'new' ones, but it seems they are either dead or old :) (read, not used in spam anymore?)
This released list has been developed from actual spam recieved since the SURBL effort started. I don't doubt there are a lot of double ups with ws.surbl.org however.
As a seperate list i would so far say, uhm, not really worth the efford, as a extra list of domains for WS, i think its worth it.
Sure - kinda what I was originally intending anyway :)
Out of 12.000 ones hit by WS i had 2 hits that were only seen by the MS list. I dont think that would be enough. Where is the date taken from? Actual recent spam mails? If so, sounds like a little outdated list ?
The list is developed from messages which slip through existing SURBL's and other custom SA rulesets - addmittedly less and less are getting through, but we do keep on top of it. It is current, but may also be a different sampling of messages to those you see.
Expiry of listings is something we're still working on, but are a little loathe to do because we've recently noticed a spate of "recycled" domain names.
Hi!
This released list has been developed from actual spam recieved since the SURBL effort started. I don't doubt there are a lot of double ups with ws.surbl.org however.
As a seperate list i would so far say, uhm, not really worth the efford, as a extra list of domains for WS, i think its worth it.
Sure - kinda what I was originally intending anyway :)
Sounds like a plan! :)
The list is developed from messages which slip through existing SURBL's and other custom SA rulesets - addmittedly less and less are getting through, but we do keep on top of it. It is current, but may also be a different sampling of messages to those you see.
Expiry of listings is something we're still working on, but are a little loathe to do because we've recently noticed a spate of "recycled" domain names.
Ok, great. It would be a nice addition to WS i think... Anyone ?
Bye, Raymond.
On Friday, July 30, 2004, 12:05:52 AM, Raymond Dijkxhoorn wrote:
This released list has been developed from actual spam recieved since the SURBL effort started. I don't doubt there are a lot of double ups with ws.surbl.org however.
As a seperate list i would so far say, uhm, not really worth the efford, as a extra list of domains for WS, i think its worth it.
Sure - kinda what I was originally intending anyway :)
Sounds like a plan! :)
The list is developed from messages which slip through existing SURBL's and other custom SA rulesets - addmittedly less and less are getting through, but we do keep on top of it. It is current, but may also be a different sampling of messages to those you see.
Expiry of listings is something we're still working on, but are a little loathe to do because we've recently noticed a spate of "recycled" domain names.
Ok, great. It would be a nice addition to WS i think... Anyone ?
Sounds good to me. Anyone else?
If so, Chris please consider pulling these into WS.
Jeff C.
-
David Groce -
David Hooton -
Eric Kolve -
Frank Ellermann -
Jeff Chan -
Mariano Absatz -
Ralph Seichter -
Ray Dzek -
Raymond Dijkxhoorn