activemq-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Baert (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (ARTEMIS-1568) Unsettled AMQP messages are lost when Receiver Link is opened on remote cluster member
Date Wed, 20 Dec 2017 19:00:00 GMT

     [ https://issues.apache.org/jira/browse/ARTEMIS-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Todd Baert updated ARTEMIS-1568:
--------------------------------
    Description: 
EDIT: This is observed in the current SNAPSHOT.

In a 2-broker cluster, if AMQP messages are sent to broker0, and then a receiver link (consumer)
is connected to broker1, the messages are removed from broker0, and attempted to be forwarded
across the cluster bridge, but are lost. broker1 shows an NPE:

{code}
ERROR [org.apache.activemq.artemis.core.server] AMQ224016: Caught exception: ActiveMQIllegalStateException[errorType=ILLEGAL_STATE
message=AMQ119029: No address configured on the Server''s Session]
	at org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1393)
[artemis-server-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1327)
[artemis-server-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1320)
[artemis-server-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.core.protocol.core.ServerSessionPacketHandler.onSessionSend(ServerSessionPacketHandler.java:661)
[artemis-server-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.core.protocol.core.ServerSessionPacketHandler.onMessagePacket(ServerSessionPacketHandler.java:264)
[artemis-server-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.utils.actors.Actor.doTask(Actor.java:33) [artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:66)
[artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42)
[artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31)
[artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:66)
[artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_151]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_151]
	at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_151]
{code}

This is similar (but not identical to) https://issues.apache.org/jira/browse/ARTEMIS-1542
which was fixed a few weeks back. In ARTEMIS-1542, the receiver links are present on both
brokers are the time the message are sent. They are now load balanced and received properly.


In *this* issue, the receiver link is created after the messages already exist in broker0.
I'm not sure if the desired behavior is for these messages simply to stay on broker0 and not
cross over to broker1, but right now, they are sent over the cluster bridge and lost forever.

In this case, it looks like the 2nd arg to ClusterConnectionBridge.beforeForward() is null.
I have attached some native AMQP client code that demonstrates this issue ( AMQP .Net Lite
client, c#):


{code:java}
using System;
using System.Collections.Generic;
using System.Threading;
using Amqp.Framing;
using Amqp.Extensions;
using Amqp.Sasl;
using Amqp.Types;

namespace Amqp.Extensions.Examples
{
    class Program
    {
        static void Main(string[] args)
        {
            // create receiver link on broker0
            Connection connection0 = new Connection(new Address("amqp://localhost:5672"));
      
            Session session0 = new Session(connection0);
            ReceiverLink receiver0 = new ReceiverLink(session0, "test", "orders");
            
            // send messages to broker0
            SenderLink sender = new SenderLink(session0, "sender", "orders");
            Message message = new Message("a message!");
            
            for (var i = 0; i < 5; i++)
            {
                sender.Send(message);
                Thread.Sleep(100);
            }

            // receive 1 of 5, works as expected...
            Message m = receiver0.Receive(TimeSpan.FromSeconds(1));
            Console.WriteLine(m.Body);
            receiver0.Accept(m);
            session0.Close();
            connection0.Close();

            // create receiver link on broker1
            Connection connection1 = new Connection(new Address("amqp://localhost:5673"));
      
            Session session1 = new Session(connection1);
            ReceiverLink receiver1 = new ReceiverLink(session1, "test", "orders");

            // these 4 messages are removed from broker0 (ack'd) but never delivered. NPE
seen in logs on broker1
            for (var i = 0; i < 4; i++)
            {   
                m = receiver1.Receive(TimeSpan.FromSeconds(1));
                if (m != null)
                {
                    Console.WriteLine(m.Body);
                    receiver1.Accept(m);
                }
            }

            sender.Close();
            session1.Close();
            connection1.Close();
        }
    }
}
{code}


  was:
In a 2-broker cluster, if AMQP messages are sent to broker0, and then a receiver link (consumer)
is connected to broker1, the messages are removed from broker0, and attempted to be forwarded
across the cluster bridge, but are lost. broker1 shows an NPE:

{code}
ERROR [org.apache.activemq.artemis.core.server] AMQ224016: Caught exception: ActiveMQIllegalStateException[errorType=ILLEGAL_STATE
message=AMQ119029: No address configured on the Server''s Session]
	at org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1393)
[artemis-server-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1327)
[artemis-server-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1320)
[artemis-server-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.core.protocol.core.ServerSessionPacketHandler.onSessionSend(ServerSessionPacketHandler.java:661)
[artemis-server-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.core.protocol.core.ServerSessionPacketHandler.onMessagePacket(ServerSessionPacketHandler.java:264)
[artemis-server-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.utils.actors.Actor.doTask(Actor.java:33) [artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:66)
[artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42)
[artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31)
[artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:66)
[artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_151]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_151]
	at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_151]
{code}

This is similar (but not identical to) https://issues.apache.org/jira/browse/ARTEMIS-1542
which was fixed a few weeks back. In ARTEMIS-1542, the receiver links are present on both
brokers are the time the message are sent. They are now load balanced and received properly.


In *this* issue, the receiver link is created after the messages already exist in broker0.
I'm not sure if the desired behavior is for these messages simply to stay on broker0 and not
cross over to broker1, but right now, they are sent over the cluster bridge and lost forever.

In this case, it looks like the 2nd arg to ClusterConnectionBridge.beforeForward() is null.
I have attached some native AMQP client code that demonstrates this issue ( AMQP .Net Lite
client, c#):


{code:java}
using System;
using System.Collections.Generic;
using System.Threading;
using Amqp.Framing;
using Amqp.Extensions;
using Amqp.Sasl;
using Amqp.Types;

namespace Amqp.Extensions.Examples
{
    class Program
    {
        static void Main(string[] args)
        {
            // create receiver link on broker0
            Connection connection0 = new Connection(new Address("amqp://localhost:5672"));
      
            Session session0 = new Session(connection0);
            ReceiverLink receiver0 = new ReceiverLink(session0, "test", "orders");
            
            // send messages to broker0
            SenderLink sender = new SenderLink(session0, "sender", "orders");
            Message message = new Message("a message!");
            
            for (var i = 0; i < 5; i++)
            {
                sender.Send(message);
                Thread.Sleep(100);
            }

            // receive 1 of 5, works as expected...
            Message m = receiver0.Receive(TimeSpan.FromSeconds(1));
            Console.WriteLine(m.Body);
            receiver0.Accept(m);
            session0.Close();
            connection0.Close();

            // create receiver link on broker1
            Connection connection1 = new Connection(new Address("amqp://localhost:5673"));
      
            Session session1 = new Session(connection1);
            ReceiverLink receiver1 = new ReceiverLink(session1, "test", "orders");

            // these 4 messages are removed from broker0 (ack'd) but never delivered. NPE
seen in logs on broker1
            for (var i = 0; i < 4; i++)
            {   
                m = receiver1.Receive(TimeSpan.FromSeconds(1));
                if (m != null)
                {
                    Console.WriteLine(m.Body);
                    receiver1.Accept(m);
                }
            }

            sender.Close();
            session1.Close();
            connection1.Close();
        }
    }
}
{code}



> Unsettled AMQP messages are lost when Receiver Link is opened on remote cluster member
> --------------------------------------------------------------------------------------
>
>                 Key: ARTEMIS-1568
>                 URL: https://issues.apache.org/jira/browse/ARTEMIS-1568
>             Project: ActiveMQ Artemis
>          Issue Type: Bug
>    Affects Versions: 2.4.0
>         Environment: Fedora 26, openJDK8, AMQP .Net Lite client
>            Reporter: Todd Baert
>
> EDIT: This is observed in the current SNAPSHOT.
> In a 2-broker cluster, if AMQP messages are sent to broker0, and then a receiver link
(consumer) is connected to broker1, the messages are removed from broker0, and attempted to
be forwarded across the cluster bridge, but are lost. broker1 shows an NPE:
> {code}
> ERROR [org.apache.activemq.artemis.core.server] AMQ224016: Caught exception: ActiveMQIllegalStateException[errorType=ILLEGAL_STATE
message=AMQ119029: No address configured on the Server''s Session]
> 	at org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1393)
[artemis-server-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
> 	at org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1327)
[artemis-server-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
> 	at org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1320)
[artemis-server-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
> 	at org.apache.activemq.artemis.core.protocol.core.ServerSessionPacketHandler.onSessionSend(ServerSessionPacketHandler.java:661)
[artemis-server-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
> 	at org.apache.activemq.artemis.core.protocol.core.ServerSessionPacketHandler.onMessagePacket(ServerSessionPacketHandler.java:264)
[artemis-server-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
> 	at org.apache.activemq.artemis.utils.actors.Actor.doTask(Actor.java:33) [artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
> 	at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:66)
[artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
> 	at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42)
[artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
> 	at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31)
[artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
> 	at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:66)
[artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_151]
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_151]
> 	at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_151]
> {code}
> This is similar (but not identical to) https://issues.apache.org/jira/browse/ARTEMIS-1542
which was fixed a few weeks back. In ARTEMIS-1542, the receiver links are present on both
brokers are the time the message are sent. They are now load balanced and received properly.

> In *this* issue, the receiver link is created after the messages already exist in broker0.
I'm not sure if the desired behavior is for these messages simply to stay on broker0 and not
cross over to broker1, but right now, they are sent over the cluster bridge and lost forever.
> In this case, it looks like the 2nd arg to ClusterConnectionBridge.beforeForward() is
null. I have attached some native AMQP client code that demonstrates this issue ( AMQP .Net
Lite client, c#):
> {code:java}
> using System;
> using System.Collections.Generic;
> using System.Threading;
> using Amqp.Framing;
> using Amqp.Extensions;
> using Amqp.Sasl;
> using Amqp.Types;
> namespace Amqp.Extensions.Examples
> {
>     class Program
>     {
>         static void Main(string[] args)
>         {
>             // create receiver link on broker0
>             Connection connection0 = new Connection(new Address("amqp://localhost:5672"));
      
>             Session session0 = new Session(connection0);
>             ReceiverLink receiver0 = new ReceiverLink(session0, "test", "orders");
>             
>             // send messages to broker0
>             SenderLink sender = new SenderLink(session0, "sender", "orders");
>             Message message = new Message("a message!");
>             
>             for (var i = 0; i < 5; i++)
>             {
>                 sender.Send(message);
>                 Thread.Sleep(100);
>             }
>             // receive 1 of 5, works as expected...
>             Message m = receiver0.Receive(TimeSpan.FromSeconds(1));
>             Console.WriteLine(m.Body);
>             receiver0.Accept(m);
>             session0.Close();
>             connection0.Close();
>             // create receiver link on broker1
>             Connection connection1 = new Connection(new Address("amqp://localhost:5673"));
      
>             Session session1 = new Session(connection1);
>             ReceiverLink receiver1 = new ReceiverLink(session1, "test", "orders");
>             // these 4 messages are removed from broker0 (ack'd) but never delivered.
NPE seen in logs on broker1
>             for (var i = 0; i < 4; i++)
>             {   
>                 m = receiver1.Receive(TimeSpan.FromSeconds(1));
>                 if (m != null)
>                 {
>                     Console.WriteLine(m.Body);
>                     receiver1.Accept(m);
>                 }
>             }
>             sender.Close();
>             session1.Close();
>             connection1.Close();
>         }
>     }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message