activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nlif <>
Subject Re: Consuming a group of messages in a single transaction
Date Fri, 28 Jul 2006 10:18:08 GMT

>> I guess I could load and send the file in chunks, using ByteMessages,
>> probably in the same manner the ActiveMQ output stream does it. So I will
>> be
>> using my own streaming solution, instead of an AMQ feature, but will gain
>> the ablility to defer commiting until I have the whole stream.
>The reason I've been suggesting splitting the file up is precisely so
>that you can commit/acknowledge the message in chunks - rather than
>deferring until you've got all of the messages. If you must defer
>until you have all of the messages then using BytesMessage or the JMS
>streams will have the same effect - the broker will need to keep them
>all around in RAM until you commit/ack. 

Ok, so what you're saying is that deferring message acknowledgement is a bad
strategy anyway, especially if the transactions are long and the files are
big. Ok, that's good advice :-)

So let's say I acknowledge file-chunks and/or other messages (that logically
belong together) one at a time, as I get them. In this case I can use the
ActiveMQ streams and maybe even non-transacted sessions, with auto-ack. But
now, if my consumer crashes while it's in the middle of such a multi-part
transmission, the messages already acknowldeged will be lost, and I can no
longer rely on the broker for redelivering them.

Therefore, maybe I should not use the broker's guaranteed delivery for this
purpose, and instead, imlpement a consumer with persistent/recoverable
cache. Every message that is acknowledged is also persisted on the consumer
side somehow (file system, database). Then, when the consumer verifies it
has all parts of the "package" (e.g. all chunks of the file and all related
messages) - it can grab them from whereever they were persisted, and forward
them together to be processed by the business tier.

BTW - I think you misunderstood my question about the ZIP file: the
intention was not compression, but to be able to pass around several things
together... but maybe it's a bad idea anyway :-)

View this message in context:
Sent from the ActiveMQ - User forum at

View raw message