activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Clebert Suconic <clebert.suco...@gmail.com>
Subject Re: [DISCUSS] AMQP Large Messages
Date Wed, 15 Nov 2017 23:33:23 GMT
It would be possible... look at
https://github.com/apache/activemq-artemis/pull/1655

I will keep the same current semantic. That is.. you would read the
whole body.. and convert in memory.
That's because I wouldn't know how to convert the body without having
it into memory.. for things like string reading.

Possible to do it without using memory.. but not very practical at
this point... we could make this a separate task.. to convert large
messages without requiring full memory read.



I would add a note to docs saying that large messages will be read
into memory before converting protocol barriers.

On Wed, Nov 15, 2017 at 6:24 PM, Michael André Pearce
<michael.andre.pearce@me.com> wrote:
> Looks good for AMQP to AMQP client.
>
> One question is in a poly protocol supported setup. Eg senders and receivers one uses
Core and AMQP.
>
> It’s not clear if this should now be possible with the new LargeMessageBody abstraction
you discuss.
>
> Cheers
> Mike
>
> Sent from my iPhone
>
>> On 16 Nov 2017, at 02:10, Clebert Suconic <clebert.suconic@gmail.com> wrote:
>>
>> I am currently implementing Large Message support for AMQP Messages.
>>
>> There's currently no support for Large Messages in AMQP, although I
>> have recently implemented a dum copy and convert, but if you send a
>> large message, say with 1 G, the whole body would be read into memory
>> before a conversion to AMQP.. what won't fit in most cases of course.
>> (even so I can always make a case where it wouldn't fit).
>>
>> LargeMessages are always converted as Core currently. With this change
>> they will be kept as AMQP all the way through, adding full support for
>> large messages, which is nice!
>>
>>
>> Anyway... the design I got so far is the following:
>>
>>
>> - LargeServerMessageImpl will delegate its implementation to another
>> class called LargeMessageBody
>>
>> - LargeMessageBody would then be used by AMQPMessage when incomplete
>> messages are sent through the wire.
>>
>> - When sending messages on the server, I will use sender.send multiple
>> times until the whole buffer is read. I'm not 100% sure qpid.Proton
>> will work fine with that approach. I will start with that, and deal
>> with further bugs if they come along.
>>
>>
>> - When receiving messages on the server. Messages are marked as
>> incomplete. So, I will create the file using the LargeMessageBody. on
>> that case I will just use the current recv method we already have
>> available on Proton multiple times. This seems to work fine already.



-- 
Clebert Suconic

Mime
View raw message