Mailing-List: contact activemq-users-help@geronimo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: activemq-users@geronimo.apache.org
Received-SPF: pass (asf.osuosl.org: domain of lists@nabble.com designates
 72.21.53.35 as permitted sender)
Message-ID: <5162189.post@talk.nabble.com>
Date: Mon, 3 Jul 2006 22:35:01 -0700 (PDT)
From: nlif <naaman@dbnet.co.il>
To: activemq-users@geronimo.apache.org
Subject: Re: Duplicating Large Messages
In-Reply-To: <ec6e67fd0607030415v3f1ac7aageb66550c009c28ce@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
References: <5139730.post@talk.nabble.com>
 <ec6e67fd0607030415v3f1ac7aageb66550c009c28ce@mail.gmail.com>


Thanks James. This is very valuable input.

Consumer prefetch - this is a great tip. I was unware of this ActiveMQ
feature. However, at this point, consumer memory is NOT my problem, since
each consumer will recieve one copy of the large message, and they should be
able to handle it. My problem is at the producer side, because it has to
duplicate the messages. 

Queue vs Topic - From your reply I understand that while using multiple
queues will indeed result in message duplication - using a topic will not.
And if using a durable topic will also take care of a scenario in which not
all of the consumers are reachable at the time of sending, while not costing
me in memory - then this may be a viable solution.  (you say that durable
topics use the disk, right?)

The reason I think I have to use selectors when using a topic, is because I
sometime want to send something to some, but not all, of my consumers, and
the recipients are determined dynamically. For example, I may need to send
message A to consumers 1,3,5 and then message B to consumers 1,3,4. The only
way I can think of is using a single Topic, adding a recipient list to the
message header, and adding selectors to the consumers. Is there any other
way? I would be happy to know.

ActiveMQ JMS Streams - Thanks for this tip! Does this mean that even when
sending a huge file, at any given moment only a small portion of it is in
memory? If so - this can be very helpful to us. Great feature!

In fact, one of the things I considered for handling very large messages,
was splitting them to smaller ones, sending them separately, and then using
an Aggregator on the consumer side (I read about this is Gregor Hohpe's
excellent "Enterprise Integration Patterns"). However, it seems to me that
when running in a cluster - an aggregator cannot work, since parts of the
message can end up in different machines on the cluster. Is there any way to
split a message to smaller chunks and then ensure they all get to the same
place, even in a cluster?

Thanks a lot for your time and insights.
Naaman


-- 
View this message in context: http://www.nabble.com/Duplicating-Large-Messages-tf1880194.html#a5162189
Sent from the ActiveMQ - User forum at Nabble.com.