activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Clebert Suconic <clebert.suco...@gmail.com>
Subject Re: [DISCUSS] AMQP Large Messages
Date Thu, 16 Nov 2017 14:31:16 GMT
There is no distinction.

It will stream if the packet is more then min Large message size on core.
Or max frame size in AMQP.

I don’t know how distinguish really large ones from the ones that are just
above the limit.

On Thu, Nov 16, 2017 at 9:11 AM Martyn Taylor <mtaylor@redhat.com> wrote:

> On Thu, Nov 16, 2017 at 2:01 PM, Clebert Suconic <
> clebert.suconic@gmail.com>
> wrote:
>
> > On Thu, Nov 16, 2017 at 6:10 AM, Martyn Taylor <mtaylor@redhat.com>
> wrote:
> > > Clebert,
> > >
> > > We need to distinguish between streamed messages and large messages.
> For
> > > the basic large message case, i.e. a single large message sent to the
> > > broker.  I agree with what you have here.
> > >
> > > Streamed large messages (i.e. messages that are received in chunks)
> > allows
> > > us to store the message without having to keep the whole thing in
> broker
> > > memory.  This get's complicated with cross protocol as:
> > >
> > > 1. Not all protocols support streamed messages.
> > Exactly.. right now in Stomp.. we read the whole body in memory before
> > sending. Just like what is in master now for AMQP.
> >
> > > 2. Cross protocol may require message conversions
> > Just like in Stomp... we read the whole message in memory
> >
> > >
> > > Both 1. and 2. would likely mean that we'd likely need to read the
> whole
> > > message into memory and I'm not sure we really want to do this?  It
> > defeats
> > > one of the main purposes of streamed messages.  (The others being
> saving
> > > client memory and reducing the amount of data to resend on connection
> > drop.)
> >
> > Most times the user will have no control over this. Say you sent a
> > 100K + 1 byte over the configured limit, String messages in Core.  the
> > client will stream it as it being large
>
> This is why I think we need to distinguish between broker large messages
> and streamed messages.  This configuration option is really for streaming
> messages.  In our docs we talk about the ability to send messages that are
> larger than the broker memory using streaming.  What happens if I do this
> and try to consume it via STOMP or another protocol that doesn't support
> streaming?
>
> > .
> >
> >
> > >
> > > For 1. I wonder whether or not we want to support this at all, or if we
> > do,
> > > make it configurable.
> > > For 2. Is it possible to transform the message via a stream?  If not
> then
> > > perhaps we treat this as a raw bytes message?
> >
> > It is possible as I said.. I think it's a separate task. I may try to
> > do it within this scope.
> >
> OK sounds good.
>
-- 
Clebert Suconic

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message