activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Strachan" <james.strac...@gmail.com>
Subject Re: SMTP Server (Apache James) spooling hints
Date Fri, 09 May 2008 12:55:32 GMT
2008/5/9 Stefano Bagnara <apache@bago.org>:
> James Strachan ha scritto:
>>
>> 2008/5/9 Stefano Bagnara <apache@bago.org>:
>>>
>>>  What does it happen under the hood when I use so many queues? Is the
>>> message fully written to disk each time I move it from a queue to another
>>> or
>>> does it simply update a reference when it belongs to the same store?
>>
>> Yeah, currently we do that.
>
> It was an "or" question, but I guess from the following sentence that you
> mean that you write the full message for each queue "move", right?

Yeah

>> Another option is to use durable topics where a message is written
>> once and all durable topic subscribers just get a kinda pointer to it.
>
> I'm not sure I understand how this would work :-(

So imagine you've 5 mailets that need to process a message. You can
write the message to 5 queues; or write the message to a single topic
and have 5 'durable topic subscribers' for each maillet. That way the
message is written once and each durable topic subscriber basically
keeps a pointer to the message.


> I liked the multiple queue solution: is there any way to limit the "writes"
> on disk with some persistent+non-persistent + longtransactions strategy?

in ActiveMQ things are either persistent; where they are written to
disk ASAP (though its up to the producer to decide if it wants to
block for it to be written completely to disk - the default - or if it
is happy to get on with something else while the write occurs) - or
they are non-persistent.

See http://activemq.apache.org/what-is-the-difference-between-persistent-and-non-persistent-delivery.html

With non-persistent we now support spooling to disk if you are running
out of RAM as another hybrid option.

The main QoS to decide really is, if you kill & restart a broker are
you happy to loose stuff?


> The fact is that my of my "most common scenario" is a input mail being
> processed through many states wihtout being altered and after 5-6 state
> changes (processor changes/queue changes) each one having 3-5
> matchers/mailets it is delivered remotely or stored locally.
> I could always store the payload to JCR so to not rewrite it multiple times,
> but I fear that even for the simple JMS message writing it once for queue
> (or even worse, once for each mailet) would be a performance issue (current
> james run an UPDATE spool set state = #newstate# where ID = #id# for status
> change and does not track persistently the "substatust" of the specific
> mailet being processed, because all the mailets in a given processor are
> processed at once for a given message).
>
>> [...]
>> https://issues.apache.org/jira/browse/INFRA-1607
>>
>> feel free to vote for it :)
>
> Done!
> I also checked on confluence administration side to see if something was
> wrong with the snippet plugin but it seems to be ok, so we'll have to wait
> for the infra team.

Yeah :(


>> As an aside - for a while I've been pondering about adding a maillet
>> support into Camel for easy Camel <-> JAMES integration.
>>
>> Something wacky to think about - which might be a bit too much Camel
>> internals for now but bear with me..
>> [... a lot of interesting technical stuff...]
>
> ATM it is very hard for me to follow you on this. I think I will have to
> read this again once I'll be more familiar with camel/activemq :-)

I thought so - never mind; if you ever get hooked on Camel come back
and read it again later and it might make a bit more sense, hopefully
:)


> But be sure that I bookmarked it and I want to try the road are trying to
> show me!!

:)
-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://open.iona.com

Mime
View raw message