qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lahiru Gunathilake <glah...@gmail.com>
Subject Re: How to handle large messages in Qpid Java client
Date Mon, 29 Nov 2010 09:51:31 GMT
FYI, during receiving a message, I can see following malformed characters in
my JMSTextMessage object,

<OUTMOD>4</OUTMOD>^D^C��^@^A^@^@^@^@^@^@

This cause the exception i have sent at the very first email, after
increasing the frame-size sometimes it works sometimes it doesn't.

Lahiru

On Mon, Nov 29, 2010 at 8:24 AM, Lahiru Gunathilake <glahiru@gmail.com>wrote:

> Hi Marnie,
>
> In IoSender and IoReceiver classes we are using a ByteBuffer to read the
> write messages, to increase the size of this buffer, I simply changed a
> constant defined in IoTransport.java file, if you look in to those three
> classes, you will get it.
>
> In IoTransport.java I changed the following line of code,
>
> private static int DEFAULT_READ_WRITE_BUFFER_SIZE = 64 * 1024;
>
> but I saw there are some other constants defined with the name of
> MAX_FRAME_SIZE but after going through the code i saw they are not
> affecting.
>
> I feel there's an issue when we send a large message in to a Queue and
> something get currupted, when do framing and send. because when I keep the
> same small frame size for read buffer and increase the size for write
> buffer, it worked for 500K messages, but couldn't get it work for more than
> 1MB large messages.
>
> Lahiru
>
> Lahiru
>
>
> On Mon, Nov 29, 2010 at 2:09 AM, Marnie McCormack <
> marnie.mccormack@googlemail.com> wrote:
>
>> Hi Lahiru,
>>
>> What exactly have you done/set for the frame size change ?
>>
>> Thanks,
>> Marnie
>>
>> On Sun, Nov 28, 2010 at 9:07 AM, Lahiru Gunathilake <glahiru@gmail.com
>> >wrote:
>>
>> > Hi all,
>> >
>> > I did a mistake when increase the framesize without increasing to
>> required
>> > size, when I increase the size I was able to read some of the messages
>> > which
>> > are in the size of 700K, but sometimes it fails.
>> >
>> >
>> > I kept on increasing the frame-size since in my setup there are messages
>> > with the size of 1.5M but I couldn't read a any of them though i
>> increase
>> > the frame-size more.
>> >
>> > other than changing in IoTransport.java, do I have to change any other
>> > constant values to sync the maximum-frame-size ?
>> >
>> > Regards
>> > Lahiru
>> >
>> > On Sat, Nov 27, 2010 at 10:34 PM, Lahiru Gunathilake <glahiru@gmail.com
>> > >wrote:
>> >
>> > > Hi Andrew,
>> > >
>> > > Thanks Andrew for the response.
>> > > I have doubled  the default frame-size which was (64 * 1024) in
>> > > IoTransport.java, but didn't worked for me. FYI, activeMQ works fine
>> in
>> > this
>> > > scenario.
>> > >
>> > > On Sat, Nov 27, 2010 at 10:22 PM, Andrew Kennedy <
>> > > andrewinternational@gmail.com> wrote:
>> > >
>> > >> Hi,
>> > >>
>> > >> The out of order frames only occur in 0-10 and with transports that
>> have
>> > >> no
>> > >> concept of sequencing, i.e. in-vm or udp, not tcp. Having said that,
>> a
>> > >> frame
>> > >> body being interpreted as a header would give those symptoms.
>> > >
>> > > I will debug and see.
>> > >
>> > > Regards
>> > > Lahiru
>> > >
>> > >> Test code that
>> > >> exercises the issue would be illuminating.
>> > >>
>> > >> Unsure if this helps?
>> > >>
>> > >> Andrew.
>> > >>
>> > >> On 26 Nov 2010 16:56, "Marnie McCormack" <
>> > marnie.mccormack@googlemail.com
>> > >> >
>> > >> wrote:
>> > >>
>> > >> Hi Lahiru,
>> > >>
>> > >> Robbie mentioned earlier when I asked him about this that he *thinks*
>> > it's
>> > >> a
>> > >> bug Andrew might have come across and possibly fixed to do with OoO
>> > >> frames.
>> > >> Certainly the exception showing that the buffer length is 1 might
>> > support
>> > >> that.
>> > >>
>> > >> I haven't had a chance to look about for this - Andrew is not about
>> > today.
>> > >>
>> > >> I'll check with him when he's back if you don't get a better reply
>> here
>> > >> first.
>> > >>
>> > >> Rgds,
>> > >> Marnie
>> > >>
>> > >> On Fri, Nov 26, 2010 at 3:27 PM, Lahiru Gunathilake <
>> glahiru@gmail.com
>> > >> >wrote:
>> > >>
>> > >>
>> > >> > Hi Marnie,
>> > >> >
>> > >> > Thanks for your prompt response. yes i tried to wait before
>> consuming,
>> > >> and
>> > >> > it wo...
>> > >>
>> > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message