activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Strachan" <>
Subject Re: SMTP Server (Apache James) spooling hints
Date Fri, 09 May 2008 12:55:32 GMT
2008/5/9 Stefano Bagnara <>:
> James Strachan ha scritto:
>> 2008/5/9 Stefano Bagnara <>:
>>>  What does it happen under the hood when I use so many queues? Is the
>>> message fully written to disk each time I move it from a queue to another
>>> or
>>> does it simply update a reference when it belongs to the same store?
>> Yeah, currently we do that.
> It was an "or" question, but I guess from the following sentence that you
> mean that you write the full message for each queue "move", right?


>> Another option is to use durable topics where a message is written
>> once and all durable topic subscribers just get a kinda pointer to it.
> I'm not sure I understand how this would work :-(

So imagine you've 5 mailets that need to process a message. You can
write the message to 5 queues; or write the message to a single topic
and have 5 'durable topic subscribers' for each maillet. That way the
message is written once and each durable topic subscriber basically
keeps a pointer to the message.

> I liked the multiple queue solution: is there any way to limit the "writes"
> on disk with some persistent+non-persistent + longtransactions strategy?

in ActiveMQ things are either persistent; where they are written to
disk ASAP (though its up to the producer to decide if it wants to
block for it to be written completely to disk - the default - or if it
is happy to get on with something else while the write occurs) - or
they are non-persistent.


With non-persistent we now support spooling to disk if you are running
out of RAM as another hybrid option.

The main QoS to decide really is, if you kill & restart a broker are
you happy to loose stuff?

> The fact is that my of my "most common scenario" is a input mail being
> processed through many states wihtout being altered and after 5-6 state
> changes (processor changes/queue changes) each one having 3-5
> matchers/mailets it is delivered remotely or stored locally.
> I could always store the payload to JCR so to not rewrite it multiple times,
> but I fear that even for the simple JMS message writing it once for queue
> (or even worse, once for each mailet) would be a performance issue (current
> james run an UPDATE spool set state = #newstate# where ID = #id# for status
> change and does not track persistently the "substatust" of the specific
> mailet being processed, because all the mailets in a given processor are
> processed at once for a given message).
>> [...]
>> feel free to vote for it :)
> Done!
> I also checked on confluence administration side to see if something was
> wrong with the snippet plugin but it seems to be ok, so we'll have to wait
> for the infra team.

Yeah :(

>> As an aside - for a while I've been pondering about adding a maillet
>> support into Camel for easy Camel <-> JAMES integration.
>> Something wacky to think about - which might be a bit too much Camel
>> internals for now but bear with me..
>> [... a lot of interesting technical stuff...]
> ATM it is very hard for me to follow you on this. I think I will have to
> read this again once I'll be more familiar with camel/activemq :-)

I thought so - never mind; if you ever get hooked on Camel come back
and read it again later and it might make a bit more sense, hopefully

> But be sure that I bookmarked it and I want to try the road are trying to
> show me!!


Open Source Integration

View raw message