activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Strachan" <james.strac...@gmail.com>
Subject Re: On message spooling
Date Thu, 11 Jan 2007 15:00:34 GMT
On 1/9/07, drvillo <f.vivoli@gmail.com> wrote:
>
> Hi all
>
> Since I'm not sure when this will come out, and in which form I'm thinking
> of implementing
> it for my specific application domain. My problem is that certain messages
> on a queue cause
> a big number of subsequent sends.  I would introduce a consumer which
> receives an
> aggregated message, store it somewhere and then periodically tries to send
> parts of it,
> looking at the size of the other queues.
> My doubt is how to implement best this try-send logic: it doesn't seem to be
> reasonable to
> do so within an onMessage(), as I would keep a transaction open for probably
> a long time
> (hours). But to implement it asynchronously would mean delegating the task
> to a further executor
> pool...
> Basically it all comes down on how to introduce timing within a JMS flow.
> Thread.sleep within a message listener doesn't seem a viable option...
>
> Any comment is more than welcome:)

FWIW spooling to disk of transient messages is now in trunk to avoid
running out of RAM when doing non-persistent messaging; I thought
thats what the subject of this thread was gonna be about :)


So if I understand you correctly, you want to receive a bunch of
messages, store them somewhere then at a later date, when its a good
time, send them right?

If so I'd write a regular transactional consumer; consume the messages
from A then, when you are ready, send them to their ultimate
destination B and do a commit.

If you are worried that queue A is too full or that there are messages
on queue A which you want to 'move out of the way for a while' you
could always move them to queue C; then forward on to queue B when you
are ready. So using queue C as a temporary staging area for things
that can wait.

e.g. take all the messages off A as fast as you can, send the 'high
priority' stuff to B, the low prority stuff to C; then when queues A
and B are small enough, you can trickle feed from C back to B again
(assuming that ordering between the high priority and low priority
messages is not vital). Just use the MBean stats to watch the queue
depths to know when you should consume more from C and move it to B.

So I think you should be able to do all of this with just a few
message consumers / producers and no need to write a separate store or
anything. I think the main use case for a separate message
database/store is when you need to combine multiple mesasges together
into a single aggregate (kinda like doing orchestration).

-- 

James
-------
http://radio.weblogs.com/0112098/

Mime
View raw message