activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Chu <eric....@gmail.com>
Subject ActiveMQ 5.2 Network of Brokers bug
Date Mon, 02 Mar 2009 00:31:00 GMT
Hi Developers,

This is my first post to a developers list so feedback is appreciated in how
I can improve how I interact with the community. (or if this is even the
proper forum to discuss a potential bug)

I have noticed inconsistent behavior between ActiveMQ 5.1 and ActiveMQ 5.2
in terms of setting up a Network of 5 Brokers and reliably getting 100% of
the messages delivered from a producer through 5 brokers to 5 consumers.
Here is my test:

*SETUP
*CentOS 4.5 Final
Java 1.5.0_11

*BROKER SETUP*
I run 5 instances of broker configured below with log level TRACE.
(ACTIVEMQ_HOME/bin/activemq xbean:BROKER<#>.xml)

BROKER1.xml
<beans
  xmlns="http://www.springframework.org/schema/beans"
  xmlns:amq="http://activemq.apache.org/schema/core"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
  http://activemq.apache.org/schema/core
http://activemq.apache.org/schema/core/activemq-core.xsd
  http://activemq.apache.org/camel/schema/spring
http://activemq.apache.org/camel/schema/spring/camel-spring.xsd">
  <broker xmlns="http://activemq.apache.org/schema/core" useJmx="true">
    <persistenceAdapter>
      <amqPersistenceAdapter directory="activemq-data1"
maxFileLength="32mb"/>
    </persistenceAdapter>
    <transportConnectors>
      <transportConnector uri="tcp://localhost:61616"/>
    </transportConnectors>
    <networkConnectors>
      <networkConnector
uri="static://(tcp://localhost:61617,tcp://localhost:61618,tcp://localhost:61619,tcp://localhost:61620)"
dynamicOnly="true" networkTTL="5"/>
    </networkConnectors>
  </broker>
</beans>

BROKER2.xml
<beans
  xmlns="http://www.springframework.org/schema/beans"
  xmlns:amq="http://activemq.apache.org/schema/core"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
  http://activemq.apache.org/schema/core
http://activemq.apache.org/schema/core/activemq-core.xsd
  http://activemq.apache.org/camel/schema/spring
http://activemq.apache.org/camel/schema/spring/camel-spring.xsd">
  <broker xmlns="http://activemq.apache.org/schema/core" useJmx="true">
    <persistenceAdapter>
      <amqPersistenceAdapter directory="activemq-data1"
maxFileLength="32mb"/>
    </persistenceAdapter>
    <transportConnectors>
      <transportConnector uri="tcp://localhost:61617"/>
    </transportConnectors>
    <networkConnectors>
      <networkConnector
uri="static://(tcp://localhost:61616,tcp://localhost:61618,tcp://localhost:61619,tcp://localhost:61620)"
dynamicOnly="true" networkTTL="5"/>
    </networkConnectors>
  </broker>
</beans>

BROKER3.xml
<beans
  xmlns="http://www.springframework.org/schema/beans"
  xmlns:amq="http://activemq.apache.org/schema/core"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
  http://activemq.apache.org/schema/core
http://activemq.apache.org/schema/core/activemq-core.xsd
  http://activemq.apache.org/camel/schema/spring
http://activemq.apache.org/camel/schema/spring/camel-spring.xsd">
  <broker xmlns="http://activemq.apache.org/schema/core" useJmx="true">
    <persistenceAdapter>
      <amqPersistenceAdapter directory="activemq-data1"
maxFileLength="32mb"/>
    </persistenceAdapter>
    <transportConnectors>
      <transportConnector uri="tcp://localhost:61618"/>
    </transportConnectors>
    <networkConnectors>
      <networkConnector
uri="static://(tcp://localhost:61616,tcp://localhost:61617,tcp://localhost:61619,tcp://localhost:61620)"
dynamicOnly="true" networkTTL="5"/>
    </networkConnectors>
  </broker>
</beans>

BROKER4.xml
<beans
  xmlns="http://www.springframework.org/schema/beans"
  xmlns:amq="http://activemq.apache.org/schema/core"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
  http://activemq.apache.org/schema/core
http://activemq.apache.org/schema/core/activemq-core.xsd
  http://activemq.apache.org/camel/schema/spring
http://activemq.apache.org/camel/schema/spring/camel-spring.xsd">
  <broker xmlns="http://activemq.apache.org/schema/core" useJmx="true">
    <persistenceAdapter>
      <amqPersistenceAdapter directory="activemq-data1"
maxFileLength="32mb"/>
    </persistenceAdapter>
    <transportConnectors>
      <transportConnector uri="tcp://localhost:61619"/>
    </transportConnectors>
    <networkConnectors>
      <networkConnector
uri="static://(tcp://localhost:61616,tcp://localhost:61617,tcp://localhost:61618,tcp://localhost:61620)"
dynamicOnly="true" networkTTL="5"/>
    </networkConnectors>
  </broker>
</beans>

BROKER5.xml
<beans
  xmlns="http://www.springframework.org/schema/beans"
  xmlns:amq="http://activemq.apache.org/schema/core"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
  http://activemq.apache.org/schema/core
http://activemq.apache.org/schema/core/activemq-core.xsd
  http://activemq.apache.org/camel/schema/spring
http://activemq.apache.org/camel/schema/spring/camel-spring.xsd">
  <broker xmlns="http://activemq.apache.org/schema/core" useJmx="true">
    <persistenceAdapter>
      <amqPersistenceAdapter directory="activemq-data1"
maxFileLength="32mb"/>
    </persistenceAdapter>
    <transportConnectors>
      <transportConnector uri="tcp://localhost:61620"/>
    </transportConnectors>
    <networkConnectors>
      <networkConnector
uri="static://(tcp://localhost:61616,tcp://localhost:61617,tcp://localhost:61618,tcp://localhost:61619)"
dynamicOnly="true" networkTTL="5"/>
    </networkConnectors>
  </broker>
</beans>

*PRODUCER* *SETUP*
I use the example producer in ACTIVEMQ_HOME/examples/ with command below
creating 1 producer that sends 10 messages:
ant producer
-Durl="failover:(tcp://localhost:61616,tcp://localhost:61617,tcp://localhost:61618,tcp://localhost:61619,tcp://localhost:61620)"
-Dsubject="test.queue" -Dmax=10

*CONSUMER SETUP*
I use the example consumer in ACTIVEMQ_HOME/examples/ with command below
creating 5 consumers:
ant consumer
-Durl="failover:(tcp://localhost:61616,tcp://localhost:61617,tcp://localhost:61618,tcp://localhost:61619,tcp://localhost:61620)"
-Dsubject="test.queue" -Dmax=100 -DsleepTime=3

If you run this test on ActiveMQ 5.1, all messages are consistently received
by the consumers. However, on ActiveMQ 5.2, messages can get orphaned on one
of the brokers. Here is what I've observed so far in debugging 5.2:

1) This symptom only occurs on a broker that doesn't have an active
consumer. (meaning a consumer and not another broker)
2) Orphaned messages have at one point traveled to every broker before
ending up on the last broker (described in #1)
3) On the final broker, the NetworkBridgeFilter logs tons of "Message all
ready routed once through this broker - ignoring" traces in the log

Thanks for taking the time to read this far. I have riddled the messaging
code with tons of debug but currently stumped as to how 5.1 knew how to
prevent a message from being dropped off at a consumerless broker. I'm
willing to take any suggestions to troubleshoot this issue further.


Best Regards,
Eric

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message