On Saturday, February 12, 2005, 3:51:53 AM, Alain Alain wrote:
Hi Jeff
I know that not all FP's are reported and there are probably no exact numbers, but it should give a good idea. Or am I wrong?
The FP reports are probably too few overall to be meaningful in terms of differentiating performance between lists. There just aren't that many, maybe a few a day on average.
Yes, but I wasn't thinking on differentiating between the lists, there are other results for. What I was thinking on was the number of FP's that exists on more than one list. This is very usefull information when combining lists. If almost no FP's do occur on more than one list (at the same time) requiring appearance on at least 2 lists would be a very safe one.
Good point. Anecdotally, FPs don't tend to appear on multiple lists very often, at least the FPs we've seen reported. This is unmeasured, just a subjective opinion. If we had some of the list data in combined form as I had proposed then we could test it better. I suppose I could just do it. ;-)
I f the reported one's are very rare, this would probably even more the case for the not reported one's. If there's a FP the chance for being reported will grow if on more than one list.
Probably the most common SURBL application, SpamAssassin, uses all the lists (though with different weights on each one) so we can assume that they're most often getting checked together, but not hitting FPs together too often.
FPs in general are sporadic, occasional errors that seem to pop up at random on different lists for different reasons. They're in the "noise" of the data and hard to classify. If they were easier to classify, we'd have an easier time eliminating them in a more formal way. We're apparently already operating at the boundaries of what can be classified in this way.
That all said, I appreciate your interest in helping to achieve lower FP rates, a goal which we should all share.
Mmm the combined lists just have to be available to someone with a big ham corpus, to test it.
Personaly knowing the results for "at least 2" or "at least 3" , would be nice. It also would be nice to know how those combination would result inside : http://www.surbl.org/permuted-hits.out.txt
Yes, the SA or SARE folks, for example, could probably write some test rules to see if combinations of SURBL lists would work better. Or I could combine them on the data side and set up some test lists.
Jeff C. -- "If it appears in hams, then don't list it."