qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robbie Gemmell <robbie.gemm...@gmail.com>
Subject Re: Sending/retrieving larger messages using Proton-J?
Date Tue, 26 Jan 2016 17:10:36 GMT
On 26 January 2016 at 15:09, Matthew Karlsen <MKarlsen@idbs.com> wrote:
> Hello all,
>
> I am presently trying to send a 1.4 meg PDF in a message and am encountering substantial
difficulties.
>
> I have tried various combinations (though my central aim is to use Proton-J). Combinations
as follows:
> Qpid Proton-J 0.11.1 to HornetQ 2.4.0 Final to Qpid Proton-J 0.11.1
> Qpid Proton-J 0.11.1 to Apache Artemis 1.2.0 to Qpid Proton-J 0.11.1
> Qpid JMS 0.7 to HornetQ 2.4.0 Final to Qpid JMS 0.7
> Qpid JMS 0.7 to Apache Artemis 1.2.0 to Qpid JMS 0.7
>
> Have also tried MQLight and SwiftMQ clients (though they are not really a viable solution
here due to their licenses).
>
> The most bytes I have managed to send and receive is ~10,000.
>
> I frequently get the two error messages below when sending (in the Artemis console window)
-- relating to the Qpid Proton library.
>
> (Note: I have tried increasing connection-ttl on the client side.)
>
> My question is this: are there any important caveats when sending large messages via
AMQP that I have somehow missed (or related caveats)?
>
> I have read the Qpid documentation, HornetQ documentation and Apache Artemis documentation
and there are some things that I think *may* be relevant, but I am not sure.
>
> The question is clearly somewhat cross-technology but I felt that this was the best place
to post the issue.
>
> I am happy to supply Wireshark output if required, though I suspect the output would
be quite voluminous.
>
> Thank you for any help you are able to offer,
> regards,
> Matthew
>
> -------------------------------------------------
>
> java.nio.BufferUnderflowException
>         at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:151)
>         at org.apache.qpid.proton.codec.DecoderImpl.readRaw(DecoderImpl.java:939)
>         at org.apache.qpid.proton.codec.BinaryType$LongBinaryEncoding.readValue(BinaryType.java:110)
>         at org.apache.qpid.proton.codec.BinaryType$LongBinaryEncoding.readValue(BinaryType.java:67)
>         at org.apache.qpid.proton.codec.DynamicTypeConstructor.readValue(DynamicTypeConstructor.java:39)
>         at org.apache.qpid.proton.codec.DecoderImpl.readObject(DecoderImpl.java:885)
>         at org.apache.qpid.proton.message.impl.MessageImpl.decode(MessageImpl.java:647)
>         at org.apache.qpid.proton.message.impl.MessageImpl.decode(MessageImpl.java:577)
>         at org.apache.qpid.proton.jms.EncodedMessage.decode(EncodedMessage.java:46)
>         at org.apache.qpid.proton.jms.JMSMappingInboundTransformer.transform(JMSMappingInboundTransformer.java:40)
>         at org.apache.activemq.artemis.core.protocol.proton.converter.ProtonMessageConverter.inboundJMSType(ProtonMessageConverter.java:57)
>         at org.apache.activemq.artemis.core.protocol.proton.converter.ProtonMessageConverter.inbound(ProtonMessageConverter.java:43)
>         at org.apache.activemq.artemis.core.protocol.proton.plug.ProtonSessionIntegrationCallback.serverSend(ProtonSessionIntegrationCallback.java:249)
>         at org.proton.plug.context.server.ProtonServerReceiverContext.onMessage(ProtonServerReceiverContext.java:116)
>         at org.proton.plug.context.AbstractConnectionContext$LocalListener.onDelivery(AbstractConnectionContext.java:250)
>         at org.proton.plug.handler.Events.dispatch(Events.java:100)
>         at org.proton.plug.handler.impl.ProtonHandlerImpl.dispatch(ProtonHandlerImpl.java:339)
>         at org.proton.plug.handler.impl.ProtonHandlerImpl.flush(ProtonHandlerImpl.java:259)
>         at org.proton.plug.handler.impl.ProtonHandlerImpl.inputBuffer(ProtonHandlerImpl.java:159)
>         at org.proton.plug.context.AbstractConnectionContext.inputBuffer(AbstractConnectionContext.java:82)
>         at org.apache.activemq.artemis.core.protocol.proton.ActiveMQProtonRemotingConnection.bufferReceived(ActiveMQProtonRemotingConnection.java:128)
>         at org.apache.activemq.artemis.core.remoting.server.impl.RemotingServiceImpl$DelegatingBufferHandler.bufferReceived(RemotingServiceImpl.java:620)
>         at org.apache.activemq.artemis.core.remoting.impl.netty.ActiveMQChannelHandler.channelRead(ActiveMQChannelHandler.java:68)
>         at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
>         at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
>         at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846)
>         at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
>         at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
>         at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
>         at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
>         at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
>         at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
>         at java.lang.Thread.run(Thread.java:745)
>
> 14:45:05,373 WARN  [org.apache.activemq.artemis.core.client] AMQ212037: Connection failure
has been detected: AMQ119014: Did not receive data from null. It is likely the client has
exited or crashed without closing its connection, or the network between the server and client
has
> failed. You also might have configured connection-ttl and client-failure-check-period
incorrectly. Please check user manual for more information. The connection will now be closed.
[code=CONNECTION_TIMEDOUT]
>


Hi Mathew,

One thing would be good, in case it matters, is if you could clarify
what specifically you are using to send/recv, as 'Proton' encompasses
a few different areas/behaviours, so the name alone isn't enough to
nail down exactly what you are doing there.

I can't say there are any caveats I can think of right now that you'd
need to be aware of in relation to sending large messages,
particularly with regard to causing BufferUnderflowExceptions on the
broker. The Qpid JMS client test suite includes some interop testing
of sending and receiving larger messages against ActiveMQ 5 (which
also uses the proton-j engine for its AMQP 1.0 support) using various
message sizes up to 50MB.

You said you tried them without success, but did you see the exact
same behaviour while using the MQLight and SwiftMQ clients?

At first glance I'd have to say the Artemis/HornetQ brokers use of the
proton-j engine is likely to be the area to look at. I have run the
JMS clients examples against Artemis, but can't say that I've sent
larger messages against it.

The second log message appears to be from Artemis core client logging,
its not clear if its related or not.

Robbie

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org


Mime
View raw message