activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From brett <bre...@ll.mit.edu>
Subject Sending large files over multipe brokers
Date Mon, 08 Nov 2010 19:32:25 GMT

I am working on a project where I will probably have two to three ActiveMQ
brokers setup in a chain where I will need to be publishing files that will
probably tens of megabytes in size. I have already been able to setup some
topic forwarding across the brokers and that all seems fine. The problem
right now is how do I transfer the files efficiently. From what I have
looked into I think there are a few ways of doing this.

It seems that with ActiveMQ you can use ActiveMQ Stream objects where you
just write to one stream from the producer side and read from another stream
on the subscribers side. I had found an example like this and was able to
get it working. The problem is that when I send a message I want it to
propagate through the brokers to the client immediately. In my test it
looked like the first broker in the chain had to receive at least 64 KB
before it passed the message on to the next broker. If I wrote more than 64
KB in one write to the stream it looked like the ActiveMQ broker made it
into multiple messages. While this does seem to work I need more control
over the size of the messages. I would like each message to proceed through
all the brokers to the consumer and not have to wait for a certain amount of
data to be collected before sending the messages on.

The second option I tried was of just sending multiple ByteMessages. In this
case I was able to control the size of the messages sent and they seemed to
propagate immediately across the brokers.

The third option I found was to use a BlobMessage. The problem here is that
I do not know if I am going to be required to send the files through
ActiveMQ  brokers and not just URL's to files that are hosted at a different
location.

So my question is what is considered the best way for transferring large
files over ActiveMQ brokers while having complete control over the size of
the messages sent? Is sending multiple ByteMessages the best solution? Is
there something about the ActiveMQ Stream messages that I overlooked that
could of given me greater control over the message size?
-- 
View this message in context: http://activemq.2283324.n4.nabble.com/Sending-large-files-over-multipe-brokers-tp3032537p3032537.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Mime
View raw message