activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefano Bagnara <apa...@bago.org>
Subject Re: SMTP Server (Apache James) spooling hints
Date Fri, 09 May 2008 16:17:38 GMT
James Strachan ha scritto:
> 2008/5/9 Stefano Bagnara <apache@bago.org>:
>> James Strachan ha scritto:
>>> Another option is to use durable topics where a message is written
>>> once and all durable topic subscribers just get a kinda pointer to it.
>> I'm not sure I understand how this would work :-(
> 
> So imagine you've 5 mailets that need to process a message. You can
> write the message to 5 queues; or write the message to a single topic
> and have 5 'durable topic subscribers' for each maillet. That way the
> message is written once and each durable topic subscriber basically
> keeps a pointer to the message.

I think I'm lost :-(

A standard scenario is that I have 3 processors:
"root" => this is where new mail (e.g: incoming from smtp) enters.
"filter" => where we decide if it is spam, local, or remote
"outgoing" => where we deliver it.

A processor in James language is a sequence of "matcher/mailets".

Currently a random mail is took from the spool, then we look at the 
current state (root, filter, outgoing) and run the processing for that 
state. The processing works like this: run the first matcher, if it 
matches run its mailet, if they didn't change the status then move to 
the second matcher, and so on. At the end of the processor the status 
have to be changed somewhere. (setting the status to "ghost" means drop 
the mail). So change the status is like moving to another queue. At each 
status change we update the status on the queue (or the whole message if 
it changed) and "unlock" it for another thread to take care of 
processing it through the new processor later.

(there is also a detail that a matcher can partially match so a copy of 
the message is created for the 2 paths to be followed, but I'm ignoring 
this at this level).

The 1:1 mapping would be to have 3 queues and do everything else like we 
do now, another "more granular" approach would be to identify queue for 
status+"matcher/mailet position" so that we have root-1, root-2, 
filter-1, filter-2, filter-3, filter-n, outgoing-1 as separate queue and 
this would give us persistence of the status in a more granular way, but 
maybe this is not needed.

You say that I can use the "durable topics" but I don't get how.
The smtpserver receive a new message and publish it in the topic, who is 
subscribed to this topic? if I subscribe each of my "processors" (root, 
filter, outgoing) how do they know that only root have to check it and 
then IF root move it to filter/outgoing status then the others will have 
to take care of it? Maybe I misunderstood and you use a combination of 
durable topics for some data and queues for some other data, but I'm 
lost on this....

I would be tempted to go for the use of the JCR to store the full 
mimemessage as soon as I receive the message and then simply put the 
"envelope" in the messaging system with a reference to the JCR so that 
the message to be moved from queue to queue (persistents) will be very 
small, but this way I'll pay the JCR storage every time even for simple 
messages I simply have to relay (that in case of AMQ Message Store I 
would simply write to the datalog), and I guess that writing to JCR does 
cost MUCH MORE than writing to ActiveMQ datalog, is this a correct guess?

Thank you,
Stefano


Mime
View raw message