activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fred Moore <fred.moor...@gmail.com>
Subject Re: best approach to transfer large File, modify content on receive
Date Wed, 03 Mar 2010 22:26:16 GMT
Hi Marc,

> We are sending files >1GB around using the ActiveMQ's BlobMessage.

...mmh you have an interesting real-life use case: can you give us a few
more details?

Things like the "side protocol" you use (FTP? WebDAV?), daily volumes, any
special other ActiveMQ tuning you performed etc

Cheers,
F.




On Wed, Mar 3, 2010 at 7:06 PM, Marc Weil <marc.weil@gmail.com> wrote:

> 100 MB isn't really that large. We are sending files >1GB around using the
> ActiveMQ's BlobMessage. Why don't you try using that instead of doing
> chunking and serialization/deserialization manually?
>
> http://activemq.apache.org/blob-messages.html
>
> Marc
>
> On Wed, Mar 3, 2010 at 6:17 AM, Cristian Botiza
> <Cristian.Botiza@endava.com>wrote:
>
> > I would look for some streaming options in ActiveMQ if you want to send
> the
> > full file. But 100 MB is quite huge...anyway see class BytesMessage. You
> may
> > try using a BufferedStream around the byte array, or read it in chuncks
> > using readBytes(byte[]).
> >
> > ________________________________________
> > From: dara kok [mrpc.cambodia@gmail.com]
> > Sent: 03 March 2010 13:05
> > To: users@activemq.apache.org
> > Subject: RE: best approach to transfer large File, modify content on
> > receive
> >
> > So you think sending the file in full size is not performance efficient?
> I
> > wonder what happens when the receiving app get the file, will the full
> > content of the file load into memory or will just part of it will get
> > loaded
> > and the rest will load on demand?
> >
> > Thanks,
> >
> >
> > Can you chunk the file? In this case you might split it between multiple
> > producers and consumers
> > and play with the number of producers/consumers and chunck size. You may
> > send and process chuncks
> > concurrently and also limit the message size.
> >
> > Just one option...
> > --
> > View this message in context:
> >
> http://old.nabble.com/best-approach-to-transfer-large-File%2C-modify-content-on-receive-tp27766358p27766785.html
> > Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >
> > The information in this email is confidential and may be legally
> > privileged. It is intended solely for the addressee. Any opinions
> expressed
> > are mine and do not necessarily represent the opinions of the Company.
> > Emails are susceptible to interference. If you are not the intended
> > recipient, any disclosure, copying, distribution or any action taken or
> > omitted to be taken in reliance on it, is strictly prohibited and may be
> > unlawful. If you have received this message in error, do not open any
> > attachments but please notify the EndavaIT Service Desk on (+44 (0)870
> 423
> > 0187), and delete this message from your system. The sender accepts no
> > responsibility for information, errors or omissions in this email, or for
> > its use or misuse, or for any act committed or omitted in connection with
> > this communication. If in doubt, please verify the authenticity of the
> > contents with the sender. Please rely on your own virus checkers as no
> > responsibility is taken by the sender for any damage rising out of any
> bug
> > or virus infection.
> >
> > Endava Limited is a company registered in England under company number
> > 5722669 whose registered office is at 125 Old Broad Street, London, EC2N
> > 1AR, United Kingdom. Endava Limited is the Endava group holding company
> and
> > does not provide any services to clients. Each of Endava Limited and its
> > subsidiaries is a separate legal entity and has no liability for another
> > such entity's acts or omissions. Please refer to the “Legal” section on
> our
> > website for a list of legal entities.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message