Hi!
I will take a look at NNTP, to see how much new code it would take to retrieve spams that way. Probably I could reuse quite a few bits from my existing POP3 code.
As for performance, I can currently handle about 60K messages per day, but I expect I could significantly speed that up. I currently check mails against SBL+XBL, and the necessary DNS lookups take up most of the elapsed time, but that wouldn't be absolutely necessary for "known bad" feed data.
Do youi allready have a local copy of those zonefiles? Would speedup processing also i guess...
I agree, it will have to be running on multiple hosts in the long term, otherwise it won't scale.
A million spams a day sounds interesting :-)
I could put a agent on one of my boxes, to help share the load, if needed.
Bye, Raymond.