activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Justin Bertram <jbert...@apache.org>
Subject Re: Artemis 2.9.0 - Basic shared storage - static discovery - Failover not happening
Date Wed, 12 Jun 2019 20:10:05 GMT
Nothing jumps out at me as problematic in either the configuration or the
logs.

The broker ships with a number of HA examples. What happens if you run the
example at examples/features/ha/non-transaction-failover? You just need to
run the command 'mvn verify' from the example directory. The example will
create and configure a master and slave instance using shared-store. If
that works then I recommend you modify the configuration step by step to
mimic your own until the example stops working. The first change I'd
recommend is modifying the <dataFolder> in the pom.xml to use
"/vagrant_data/data". Try that out and let me know how it goes.


Justin

On Wed, Jun 12, 2019 at 10:15 AM hannibal <hannibal80@gmail.com> wrote:

> Hi all,
> I first tried with Artemis 2.7.0 but I didn't make it, now I'm trying with
> Artemis 2.9.0, still looking for some luck.
>
> What I want to achieve is a basic shared storage master/slave cluster with
> static discovery.
>
> I have a java client that connects to the master. Java code is provided
> below.
>
> Test steps:
> 1. start master server
> 2. start slave server
> 3. start client app
> 4. kill master server
>
> The Java app gives exception and stop sending/receiving messages. Failover
> is not happening.
> Any help would be very appreciated.
>
> Thanks,
> Lorenzo
>
> ______________
>
> Here Java client code:
>
> static final String ARTEMIS_CLUSTER_URL_1M =
>
> "tcp://artemis-1M:61616?ha=true&retryInterval=1000&retryIntervalMultiplier=1.0&reconnectAttempts=-1";
> static final String ARTEMIS_CLUSER_USERNAME = "ADMIN.USER";
> static final String ARTEMIS_CLUSTER_PASSWORD = "XxXxX";
> int CONS_THREADS = 20;
> int PROD_THREADS = 1;
>
> ActiveMQConnectionFactory connectionFactory = new
> ActiveMQConnectionFactory(ARTEMIS_CLUSTER_URL_1M, ARTEMIS_CLUSER_USERNAME,
> ARTEMIS_CLUSTER_PASSWORD);
>
> Connection consConn = connectionFactory.createConnection();
> consConn.start();
>
> Connection prodConn = connectionFactory.createConnection();
> prodConn.start();
>
> ExecutorService executor = Executors.newFixedThreadPool(CONS_THREADS);
>
> for (int i=0 ; i < CONS_THREADS; i++) {
>     executor.execute(new ArtemisConsumer(consConn));
> }
>
> for (int i=0 ; i < PROD_THREADS; i++) {
>     executor.execute(new ArtemisProducer(prodConn));
> }
>
> ...
>
> public class ArtemisConsumer implements Runnable {
>     private Connection connection = null;
>
>     public ArtemisConsumer(Connection connection) {
>         this.connection = connection;
>     }
>
>     public void run() {
>
>         try {
>             Session session = connection.createSession(false,
> Session.AUTO_ACKNOWLEDGE);
>             Queue queue = session.createQueue("jms.artemis-test-queue");
>             MessageConsumer consumer = session.createConsumer(queue);
>             consumer.setMessageListener(new SimpleMessageListner(session));
>         } catch (Exception e) {
>             System.out.println("Caught exception!" + e);
>             e.printStackTrace();
>         }
>
>     }
> }
>
> ...
>
> public class ArtemisProducer implements Runnable{
>     private static String MESSAGE;
>     private static Connection connection = null;
>     private int name;
>
>
>     public ArtemisProducer(Connection connection) {
>         this.connection = connection;
>         if(MESSAGE==null) {
>             MESSAGE = createMessage();
>         }
>     }
>
>     public void run() {
>
>         try {
>
>             Session session = connection.createSession(false, Session
>                     .AUTO_ACKNOWLEDGE);
>             Queue queue = session.createQueue("jms.artemis-test-queue");
>             MessageProducer producer = session.createProducer(queue);
>             producer.setDeliveryMode(DeliveryMode.PERSISTENT);
>             int p=0;
>
>             while (p < 50000) {
>                 TextMessage message = session.createTextMessage(MESSAGE);
>                 producer.send(message);
>             }
>
>             producer.close();
>
>             session.close();
>
>         } catch (Exception e) {
>             System.out.println("Caught exception! " + e);
>             e.printStackTrace();
>         }
>     }
>
>     private String createMessage() {
>         byte[] array = new byte[runCfg.msgSize*1024]; // length is bounded
> by 7
>         new Random().nextBytes(array);
>
>         return new String(array, Charset.forName("UTF-8"));
>     }
>
>
> ...
>
> public class SimpleMessageListner implements MessageListener{
>
>     private Session session;
>
>     public SimpleMessageListner(Session session) {
>         this.session = session;
>     }
>
>     public SimpleMessageListner(Session session) {
>         this.session = session;
>     }
>
>     @Override
>     public void onMessage(Message message) {
>         ...
>     }
> }
>
> ...
>
> Here the client exception:
>
> Caught exception! javax.jms.JMSException: AMQ219014: Timed out after
> waiting
> 30,000 ms for response when sending packet 71
> javax.jms.JMSException: AMQ219014: Timed out after waiting 30,000 ms for
> response when sending packet 71
>         at
>
> org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:454)
>         at
>
> org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:358)
>         at
>
> org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQSessionContext.sendFullMessage(ActiveMQSessionContext.java:554)
>         at
>
> org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.sendRegularMessage(ClientProducerImpl.java:303)
>         at
>
> org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.doSend(ClientProducerImpl.java:275)
>         at
>
> org.apache.activemq.artemis.core.client.impl.ClientProducerImpl.send(ClientProducerImpl.java:128)
>         at
>
> org.apache.activemq.artemis.jms.client.ActiveMQMessageProducer.doSendx(ActiveMQMessageProducer.java:485)
>         at
>
> org.apache.activemq.artemis.jms.client.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:195)
>         at
>
> com.activemq.test.perf.artemis.producer.ArtemisProducer.run(ArtemisProducer.java:39)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
> Source)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
>         at java.lang.Thread.run(Unknown Source)
> Caused by:
> ActiveMQConnectionTimedOutException[errorType=CONNECTION_TIMEDOUT
> message=AMQ219014: Timed out after waiting 30,000 ms for response when
> sending packet 71]
>         ... 12 more
>
>
> Here master configuration:
>
> <?xml version='1.0'?>
>
> <configuration xmlns="urn:activemq"
>                xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>                xmlns:xi="http://www.w3.org/2001/XInclude"
>                xsi:schemaLocation="urn:activemq
> /schema/artemis-configuration.xsd">
>
>    <core xmlns="urn:activemq:core"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>          xsi:schemaLocation="urn:activemq:core ">
>
>       <name>artemis-1M</name>
>
>
>       <persistence-enabled>true</persistence-enabled>
>
>       <journal-type>ASYNCIO</journal-type>
>
>       <paging-directory>/vagrant_data/data/paging</paging-directory>
>
>       <bindings-directory>/vagrant_data/data/bindings</bindings-directory>
>
>       <journal-directory>/vagrant_data/data/journal</journal-directory>
>
>
>
> <large-messages-directory>/vagrant_data/data/large-messages</large-messages-directory>
>
>       <journal-datasync>true</journal-datasync>
>
>       <journal-min-files>2</journal-min-files>
>
>       <journal-pool-files>10</journal-pool-files>
>
>       <journal-file-size>10M</journal-file-size>
>
>       <journal-buffer-timeout>16000</journal-buffer-timeout>
>
>
>
>       <journal-max-io>4096</journal-max-io>
>
>
>       <disk-scan-period>5000</disk-scan-period>
>
>
>       <max-disk-usage>90</max-disk-usage>
>
>
>       <critical-analyzer>true</critical-analyzer>
>
>       <critical-analyzer-timeout>120000</critical-analyzer-timeout>
>
>
> <critical-analyzer-check-period>60000</critical-analyzer-check-period>
>
>       <critical-analyzer-policy>HALT</critical-analyzer-policy>
>
>           <ha-policy>
>                 <shared-store>
>                         <master>
>
> <failover-on-shutdown>true</failover-on-shutdown>
>                         </master>
>                 </shared-store>
>           </ha-policy>
>
>
>
>
>         <cluster-user>ADMIN.USER</cluster-user>
>         <cluster-password>XxXxX</cluster-password>
>
>         <cluster-connections>
>                 <cluster-connection name="artemis-cluster">
>                         <connector-ref>artemis-1M</connector-ref>
>
>                         <static-connectors
> allow-direct-connections-only="true">
>
>                                 <connector-ref>artemis-1S</connector-ref>
>
>                         </static-connectors>
>                 </cluster-connection>
>         </cluster-connections>
>
>         <connectors>
>                 <connector
> name="artemis-1M">tcp://artemis-1M:61616</connector>
>                 <connector
> name="artemis-1S">tcp://artemis-1S:61616</connector>
>
>         </connectors>
>
>       <acceptors>
>
>                  <acceptor name="artemis">tcp://0.0.0.0:61616</acceptor>
>
>       </acceptors>
>
>
>       <security-settings>
>          <security-setting match="#">
>             <permission type="createNonDurableQueue" roles="amq"/>
>             <permission type="deleteNonDurableQueue" roles="amq"/>
>             <permission type="createDurableQueue" roles="amq"/>
>             <permission type="deleteDurableQueue" roles="amq"/>
>             <permission type="createAddress" roles="amq"/>
>             <permission type="deleteAddress" roles="amq"/>
>             <permission type="consume" roles="amq"/>
>             <permission type="browse" roles="amq"/>
>             <permission type="send" roles="amq"/>
>
>             <permission type="manage" roles="amq"/>
>          </security-setting>
>       </security-settings>
>
>       <address-settings>
>
>          <address-setting match="activemq.management#">
>             <dead-letter-address>DLQ</dead-letter-address>
>             <expiry-address>ExpiryQueue</expiry-address>
>             <redelivery-delay>0</redelivery-delay>
>
>             <max-size-bytes>-1</max-size-bytes>
>
> <message-counter-history-day-limit>10</message-counter-history-day-limit>
>             <address-full-policy>PAGE</address-full-policy>
>             <auto-create-queues>true</auto-create-queues>
>             <auto-create-addresses>true</auto-create-addresses>
>             <auto-create-jms-queues>true</auto-create-jms-queues>
>             <auto-create-jms-topics>true</auto-create-jms-topics>
>          </address-setting>
>
>          <address-setting match="#">
>             <dead-letter-address>DLQ</dead-letter-address>
>             <expiry-address>ExpiryQueue</expiry-address>
>             <redelivery-delay>0</redelivery-delay>
>
>             <max-size-bytes>-1</max-size-bytes>
>
> <message-counter-history-day-limit>10</message-counter-history-day-limit>
>             <address-full-policy>PAGE</address-full-policy>
>             <auto-create-queues>true</auto-create-queues>
>             <auto-create-addresses>true</auto-create-addresses>
>             <auto-create-jms-queues>true</auto-create-jms-queues>
>             <auto-create-jms-topics>true</auto-create-jms-topics>
>          </address-setting>
>
>                 <address-setting match="jms.#">
>                         <redistribution-delay>1000</redistribution-delay>
>                 </address-setting>
>
>       </address-settings>
>
>       <addresses>
>          <address name="DLQ">
>             <anycast>
>                <queue name="DLQ" />
>             </anycast>
>          </address>
>          <address name="ExpiryQueue">
>             <anycast>
>                <queue name="ExpiryQueue" />
>             </anycast>
>          </address>
>
>       </addresses>
>
>
>
>       <broker-plugins>
>          <broker-plugin
>
> class-name="org.apache.activemq.artemis.core.server.plugin.impl.LoggingActiveMQServerPlugin">
>             <property key="LOG_ALL_EVENTS" value="false"/>
>             <property key="LOG_CONNECTION_EVENTS" value="true"/>
>             <property key="LOG_SESSION_EVENTS" value="true"/>
>             <property key="LOG_CONSUMER_EVENTS" value="false"/>
>             <property key="LOG_DELIVERING_EVENTS" value="false"/>
>             <property key="LOG_SENDING_EVENTS" value="false"/>
>             <property key="LOG_INTERNAL_EVENTS" value="true"/>
>          </broker-plugin>
>       </broker-plugins>
>
>
>    </core>
> </configuration>
>
> ...
>
> Here slave configuration:
>
> <?xml version='1.0'?>
>
> <configuration xmlns="urn:activemq"
>                xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>                xmlns:xi="http://www.w3.org/2001/XInclude"
>                xsi:schemaLocation="urn:activemq
> /schema/artemis-configuration.xsd">
>
>    <core xmlns="urn:activemq:core"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>          xsi:schemaLocation="urn:activemq:core ">
>
>       <name>artemis-1S</name>
>
>
>       <persistence-enabled>true</persistence-enabled>
>
>       <journal-type>ASYNCIO</journal-type>
>
>       <paging-directory>/vagrant_data/data/paging</paging-directory>
>
>       <bindings-directory>/vagrant_data/data/bindings</bindings-directory>
>
>       <journal-directory>/vagrant_data/data/journal</journal-directory>
>
>
>
> <large-messages-directory>/vagrant_data/data/large-messages</large-messages-directory>
>
>       <journal-datasync>true</journal-datasync>
>
>       <journal-min-files>2</journal-min-files>
>
>       <journal-pool-files>10</journal-pool-files>
>
>       <journal-file-size>10M</journal-file-size>
>
>       <journal-buffer-timeout>16000</journal-buffer-timeout>
>
>
>
>       <journal-max-io>4096</journal-max-io>
>
>
>       <disk-scan-period>5000</disk-scan-period>
>
>
>       <max-disk-usage>90</max-disk-usage>
>
>
>       <critical-analyzer>true</critical-analyzer>
>
>       <critical-analyzer-timeout>120000</critical-analyzer-timeout>
>
>
> <critical-analyzer-check-period>60000</critical-analyzer-check-period>
>
>       <critical-analyzer-policy>HALT</critical-analyzer-policy>
>
>           <ha-policy>
>                 <shared-store>
>                   <slave>
>                         <failover-on-shutdown>true</failover-on-shutdown>
>                         <allow-failback>true</allow-failback>
>                   </slave>
>                 </shared-store>
>           </ha-policy>
>
>
>
>         <cluster-user>ADMIN.USER</cluster-user>
>         <cluster-password>XxXxX</cluster-password>
>
>         <cluster-connections>
>                 <cluster-connection name="artemis-cluster">
>                         <connector-ref>artemis-1S</connector-ref>
>                         <static-connectors
> allow-direct-connections-only="true">
>                                 <connector-ref>artemis-1M</connector-ref>
>
>
>                         </static-connectors>
>                 </cluster-connection>
>         </cluster-connections>
>
>         <connectors>
>                 <connector
> name="artemis-1M">tcp://artemis-1M:61616</connector>
>                 <connector
> name="artemis-1S">tcp://artemis-1S:61616</connector>
>
>         </connectors>
>
>
>
>       <acceptors>
>
>          <acceptor name="artemis">tcp://0.0.0.0:61616</acceptor>
>
>       </acceptors>
>
>
>       <security-settings>
>          <security-setting match="#">
>             <permission type="createNonDurableQueue" roles="amq"/>
>             <permission type="deleteNonDurableQueue" roles="amq"/>
>             <permission type="createDurableQueue" roles="amq"/>
>             <permission type="deleteDurableQueue" roles="amq"/>
>             <permission type="createAddress" roles="amq"/>
>             <permission type="deleteAddress" roles="amq"/>
>             <permission type="consume" roles="amq"/>
>             <permission type="browse" roles="amq"/>
>             <permission type="send" roles="amq"/>
>
>             <permission type="manage" roles="amq"/>
>          </security-setting>
>       </security-settings>
>
>       <address-settings>
>
>          <address-setting match="activemq.management#">
>             <dead-letter-address>DLQ</dead-letter-address>
>             <expiry-address>ExpiryQueue</expiry-address>
>             <redelivery-delay>0</redelivery-delay>
>
>             <max-size-bytes>-1</max-size-bytes>
>
> <message-counter-history-day-limit>10</message-counter-history-day-limit>
>             <address-full-policy>PAGE</address-full-policy>
>             <auto-create-queues>true</auto-create-queues>
>             <auto-create-addresses>true</auto-create-addresses>
>             <auto-create-jms-queues>true</auto-create-jms-queues>
>             <auto-create-jms-topics>true</auto-create-jms-topics>
>          </address-setting>
>
>          <address-setting match="#">
>             <dead-letter-address>DLQ</dead-letter-address>
>             <expiry-address>ExpiryQueue</expiry-address>
>             <redelivery-delay>0</redelivery-delay>
>
>
>             <max-size-bytes>-1</max-size-bytes>
>
> <message-counter-history-day-limit>10</message-counter-history-day-limit>
>             <address-full-policy>PAGE</address-full-policy>
>             <auto-create-queues>true</auto-create-queues>
>             <auto-create-addresses>true</auto-create-addresses>
>             <auto-create-jms-queues>true</auto-create-jms-queues>
>             <auto-create-jms-topics>true</auto-create-jms-topics>
>          </address-setting>
>
>                 <address-setting match="jms.#">
>                         <redistribution-delay>1000</redistribution-delay>
>                 </address-setting>
>
>       </address-settings>
>
>       <addresses>
>          <address name="DLQ">
>             <anycast>
>                <queue name="DLQ" />
>             </anycast>
>          </address>
>          <address name="ExpiryQueue">
>             <anycast>
>                <queue name="ExpiryQueue" />
>             </anycast>
>          </address>
>
>       </addresses>
>
>
>
>       <broker-plugins>
>          <broker-plugin
>
> class-name="org.apache.activemq.artemis.core.server.plugin.impl.LoggingActiveMQServerPlugin">
>             <property key="LOG_ALL_EVENTS" value="false"/>
>             <property key="LOG_CONNECTION_EVENTS" value="true"/>
>             <property key="LOG_SESSION_EVENTS" value="true"/>
>             <property key="LOG_CONSUMER_EVENTS" value="false"/>
>             <property key="LOG_DELIVERING_EVENTS" value="false"/>
>             <property key="LOG_SENDING_EVENTS" value="false"/>
>             <property key="LOG_INTERNAL_EVENTS" value="true"/>
>          </broker-plugin>
>       </broker-plugins>
>
>
>    </core>
> </configuration>
>
> Here master server logs:
>
> 2019-06-12 08:06:20,573 INFO
> [org.apache.activemq.artemis.integration.bootstrap] AMQ101000: Starting
> ActiveMQ Artemis Server
> 2019-06-12 08:06:20,641 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221000: live Message Broker is starting with configuration Broker
> Configuration
>
> (clustered=true,journalDirectory=/vagrant_data/data/journal,bindingsDirectory=/vagrant_data/data/bindings,largeMessagesDirectory=/vagrant_data/data/large-messages,pagingDirectory=/vagrant_data/data/paging)
> 2019-06-12 08:06:20,644 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221006: Waiting to obtain live lock
> 2019-06-12 08:06:20,648 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221054: libaio was found but the filesystem does not support AIO.
> Switching the configuration into NIO. Journal path:
> /vagrant_data/data/journal
> 2019-06-12 08:06:20,700 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221013: Using NIO Journal
> 2019-06-12 08:06:20,763 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221057: Global Max Size is being adjusted to 1/2 of the JVM max size
> (-Xmx). being defined as 1,073,741,824
> 2019-06-12 08:06:20,935 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221043: Protocol module found: [artemis-server]. Adding protocol support
> for: CORE
> 2019-06-12 08:06:20,942 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221043: Protocol module found: [artemis-amqp-protocol]. Adding protocol
> support for: AMQP
> 2019-06-12 08:06:20,947 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221043: Protocol module found: [artemis-hornetq-protocol]. Adding
> protocol support for: HORNETQ
> 2019-06-12 08:06:20,962 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221043: Protocol module found: [artemis-mqtt-protocol]. Adding protocol
> support for: MQTT
> 2019-06-12 08:06:20,968 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221043: Protocol module found: [artemis-openwire-protocol]. Adding
> protocol support for: OPENWIRE
> 2019-06-12 08:06:20,971 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221043: Protocol module found: [artemis-stomp-protocol]. Adding protocol
> support for: STOMP
> 2019-06-12 08:06:21,129 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221034: Waiting indefinitely to obtain live lock
> 2019-06-12 08:06:21,129 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221035: Live Server Obtained live lock
> 2019-06-12 08:06:23,234 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221080: Deploying address DLQ supporting [ANYCAST]
> 2019-06-12 08:06:23,236 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221003: Deploying ANYCAST queue DLQ on address DLQ
> 2019-06-12 08:06:23,237 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221080: Deploying address ExpiryQueue supporting [ANYCAST]
> 2019-06-12 08:06:23,237 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221003: Deploying ANYCAST queue ExpiryQueue on address ExpiryQueue
> 2019-06-12 08:06:23,629 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221020: Started EPOLL Acceptor at 0.0.0.0:61616 for protocols
> [CORE,MQTT,AMQP,HORNETQ,STOMP,OPENWIRE]
> 2019-06-12 08:06:23,631 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221007: Server is now live
> 2019-06-12 08:06:23,632 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221001: Apache ActiveMQ Artemis Message Broker version 2.9.0
> [artemis-1M,
> nodeID=46af3acc-7654-11e9-a97e-08002741bcc2]
> 2019-06-12 08:06:24,233 INFO
> [org.apache.activemq.hawtio.branding.PluginContextListener] Initialized
> activemq-branding plugin
> 2019-06-12 08:06:24,441 INFO
> [org.apache.activemq.hawtio.plugin.PluginContextListener] Initialized
> artemis-plugin plugin
> 2019-06-12 08:06:25,462 INFO  [io.hawt.HawtioContextListener] Initialising
> hawtio services
> 2019-06-12 08:06:25,596 INFO  [io.hawt.system.ConfigManager] Configuration
> will be discovered via system properties
> 2019-06-12 08:06:25,599 INFO  [io.hawt.jmx.JmxTreeWatcher] Welcome to
> hawtio
> 1.5.5 : http://hawt.io/ : Don't cha wish your console was hawt like me?
> ;-)
> 2019-06-12 08:06:25,644 INFO  [io.hawt.jmx.UploadManager] Using file upload
> directory: /var/lib/artemis/tmp/uploads
> 2019-06-12 08:06:25,736 INFO  [io.hawt.web.AuthenticationFilter] Starting
> hawtio authentication filter, JAAS realm: "activemq" authorized role(s):
> "amq" role principal classes:
> "org.apache.activemq.artemis.spi.core.security.jaas.RolePrincipal"
> 2019-06-12 08:06:25,820 INFO  [io.hawt.web.JolokiaConfiguredAgentServlet]
> Jolokia overridden property: [key=policyLocation,
> value=file:/var/lib/artemis/etc/jolokia-access.xml]
> 2019-06-12 08:06:25,888 INFO  [io.hawt.web.RBACMBeanInvoker] Using MBean
> [hawtio:type=security,area=jmx,rank=0,name=HawtioDummyJMXSecurity] for role
> based access control
> 2019-06-12 08:06:26,302 INFO  [io.hawt.system.ProxyWhitelist] Initial proxy
> whitelist: [localhost, 127.0.0.1, 10.0.2.15, 192.168.1.10, artemis-1M]
> 2019-06-12 08:06:27,479 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP
> Server started at http://192.168.1.10:8161
> 2019-06-12 08:06:27,480 INFO  [org.apache.activemq.artemis] AMQ241002:
> Artemis Jolokia REST API available at
> http://192.168.1.10:8161/console/jolokia
> 2019-06-12 08:06:27,481 INFO  [org.apache.activemq.artemis] AMQ241004:
> Artemis Console available at http://192.168.1.10:8161/console
> 2019-06-12 08:06:36,106 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841000: created
> connection: RemotingConnectionImpl [ID=4d972f93, clientID=null,
> nodeID=46af3acc-7654-11e9-a97e-08002741bcc2,
>
> transportConnection=org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@7844fa34
> [ID=4d972f93,
> local= /192.168.1.10:61616, remote=/192.168.1.20:40612]]
> 2019-06-12 08:06:36,106 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841000: created
> connection: RemotingConnectionImpl [ID=1281a797, clientID=null,
> nodeID=46af3acc-7654-11e9-a97e-08002741bcc2,
>
> transportConnection=org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@7af04c08
> [ID=1281a797,
> local= /192.168.1.10:61616, remote=/192.168.1.20:40610]]
> 2019-06-12 08:06:39,704 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841000: created
> connection: RemotingConnectionImpl [ID=ef541e18, clientID=null,
> nodeID=46af3acc-7654-11e9-a97e-08002741bcc2,
>
> transportConnection=org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@46cd9da2
> [ID=ef541e18,
> local= /192.168.1.10:61616, remote=/192.168.1.20:40614]]
> 2019-06-12 08:06:52,584 INFO  [io.hawt.web.LoginServlet] hawtio login is
> using 1800 sec. HttpSession timeout
> 2019-06-12 08:07:14,993 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841000: created
> connection: RemotingConnectionImpl [ID=8e78eb87, clientID=null,
> nodeID=46af3acc-7654-11e9-a97e-08002741bcc2,
>
> transportConnection=org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@41dbb1cc
> [ID=8e78eb87,
> local= /192.168.1.10:61616, remote=/192.168.1.1:61125]]
> 2019-06-12 08:07:15,087 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created
> session name: c357e5d5-8d23-11e9-87cd-0a0027000007, session connectionID:
> 8e78eb87
> 2019-06-12 08:07:15,116 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841004: added
> session metadata for session name : c357e5d5-8d23-11e9-87cd-0a0027000007,
> key: jms-session, data:
> 2019-06-12 08:07:15,132 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created
> session name: c3679d46-8d23-11e9-87cd-0a0027000007, session connectionID:
> 8e78eb87
> 2019-06-12 08:07:15,150 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841004: added
> session metadata for session name : c3679d46-8d23-11e9-87cd-0a0027000007,
> key: jms-session, data:
> 2019-06-12 08:07:15,166 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created
> session name: c36d90b7-8d23-11e9-87cd-0a0027000007, session connectionID:
> 8e78eb87
> 2019-06-12 08:07:15,174 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841004: added
> session metadata for session name : c36d90b7-8d23-11e9-87cd-0a0027000007,
> key: jms-session, data:
> 2019-06-12 08:07:15,178 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created
> session name: c36fdaa8-8d23-11e9-87cd-0a0027000007, session connectionID:
> 8e78eb87
> 2019-06-12 08:07:15,183 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841004: added
> session metadata for session name : c36fdaa8-8d23-11e9-87cd-0a0027000007,
> key: jms-session, data:
> 2019-06-12 08:07:15,202 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created
> session name: c3722499-8d23-11e9-87cd-0a0027000007, session connectionID:
> 8e78eb87
> 2019-06-12 08:07:15,215 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841004: added
> session metadata for session name : c3722499-8d23-11e9-87cd-0a0027000007,
> key: jms-session, data:
> 2019-06-12 08:07:15,221 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created
> session name: c376434a-8d23-11e9-87cd-0a0027000007, session connectionID:
> 8e78eb87
> 2019-06-12 08:07:15,227 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841004: added
> session metadata for session name : c376434a-8d23-11e9-87cd-0a0027000007,
> key: jms-session, data:
> 2019-06-12 08:07:15,236 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created
> session name: c378b44b-8d23-11e9-87cd-0a0027000007, session connectionID:
> 8e78eb87
> 2019-06-12 08:07:15,246 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841004: added
> session metadata for session name : c378b44b-8d23-11e9-87cd-0a0027000007,
> key: jms-session, data:
> 2019-06-12 08:07:15,258 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created
> session name: c37bc18c-8d23-11e9-87cd-0a0027000007, session connectionID:
> 8e78eb87
> 2019-06-12 08:07:15,266 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841004: added
> session metadata for session name : c37bc18c-8d23-11e9-87cd-0a0027000007,
> key: jms-session, data:
> 2019-06-12 08:07:15,273 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created
> session name: c37e0b7d-8d23-11e9-87cd-0a0027000007, session connectionID:
> 8e78eb87
> 2019-06-12 08:07:15,281 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841004: added
> session metadata for session name : c37e0b7d-8d23-11e9-87cd-0a0027000007,
> key: jms-session, data:
> 2019-06-12 08:07:15,285 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created
> session name: c3802e5e-8d23-11e9-87cd-0a0027000007, session connectionID:
> 8e78eb87
> 2019-06-12 08:07:15,288 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841004: added
> session metadata for session name : c3802e5e-8d23-11e9-87cd-0a0027000007,
> key: jms-session, data:
> 2019-06-12 08:07:15,297 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created
> session name: c382031f-8d23-11e9-87cd-0a0027000007, session connectionID:
> 8e78eb87
> 2019-06-12 08:07:15,301 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841004: added
> session metadata for session name : c382031f-8d23-11e9-87cd-0a0027000007,
> key: jms-session, data:
> 2019-06-12 08:07:15,328 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created
> session name: c38648e0-8d23-11e9-87cd-0a0027000007, session connectionID:
> 8e78eb87
> 2019-06-12 08:07:15,353 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841004: added
> session metadata for session name : c38648e0-8d23-11e9-87cd-0a0027000007,
> key: jms-session, data:
> 2019-06-12 08:07:15,388 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created
> session name: c38cffa1-8d23-11e9-87cd-0a0027000007, session connectionID:
> 8e78eb87
> 2019-06-12 08:07:15,395 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841004: added
> session metadata for session name : c38cffa1-8d23-11e9-87cd-0a0027000007,
> key: jms-session, data:
> 2019-06-12 08:07:15,401 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created
> session name: c391e1a2-8d23-11e9-87cd-0a0027000007, session connectionID:
> 8e78eb87
> 2019-06-12 08:07:15,405 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841004: added
> session metadata for session name : c391e1a2-8d23-11e9-87cd-0a0027000007,
> key: jms-session, data:
> 2019-06-12 08:07:15,413 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created
> session name: c3938f53-8d23-11e9-87cd-0a0027000007, session connectionID:
> 8e78eb87
> 2019-06-12 08:07:15,422 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841004: added
> session metadata for session name : c3938f53-8d23-11e9-87cd-0a0027000007,
> key: jms-session, data:
> 2019-06-12 08:07:15,425 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created
> session name: c3958b24-8d23-11e9-87cd-0a0027000007, session connectionID:
> 8e78eb87
> 2019-06-12 08:07:15,432 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841004: added
> session metadata for session name : c3958b24-8d23-11e9-87cd-0a0027000007,
> key: jms-session, data:
> 2019-06-12 08:07:15,439 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created
> session name: c397ae05-8d23-11e9-87cd-0a0027000007, session connectionID:
> 8e78eb87
> 2019-06-12 08:07:15,446 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841004: added
> session metadata for session name : c397ae05-8d23-11e9-87cd-0a0027000007,
> key: jms-session, data:
> 2019-06-12 08:07:15,472 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created
> session name: c39982c6-8d23-11e9-87cd-0a0027000007, session connectionID:
> 8e78eb87
> 2019-06-12 08:07:15,475 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841004: added
> session metadata for session name : c39982c6-8d23-11e9-87cd-0a0027000007,
> key: jms-session, data:
> 2019-06-12 08:07:15,483 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created
> session name: c39e16a7-8d23-11e9-87cd-0a0027000007, session connectionID:
> 8e78eb87
> 2019-06-12 08:07:15,488 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841004: added
> session metadata for session name : c39e16a7-8d23-11e9-87cd-0a0027000007,
> key: jms-session, data:
> 2019-06-12 08:07:15,495 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created
> session name: c3a01278-8d23-11e9-87cd-0a0027000007, session connectionID:
> 8e78eb87
> 2019-06-12 08:07:15,500 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841004: added
> session metadata for session name : c3a01278-8d23-11e9-87cd-0a0027000007,
> key: jms-session, data:
> 2019-06-12 08:07:15,505 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created
> session name: c3a1c029-8d23-11e9-87cd-0a0027000007, session connectionID:
> 8e78eb87
> 2019-06-12 08:07:15,509 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841004: added
> session metadata for session name : c3a1c029-8d23-11e9-87cd-0a0027000007,
> key: jms-session, data:
> 2019-06-12 08:07:25,153 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841000: created
> connection: RemotingConnectionImpl [ID=a77a2904, clientID=null,
> nodeID=46af3acc-7654-11e9-a97e-08002741bcc2,
>
> transportConnection=org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@58a54051
> [ID=a77a2904,
> local= /192.168.1.10:61616, remote=/192.168.1.1:61130]]
> 2019-06-12 08:07:25,163 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created
> session name: c962fc9c-8d23-11e9-87cd-0a0027000007, session connectionID:
> a77a2904
> 2019-06-12 08:07:25,165 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841004: added
> session metadata for session name : c962fc9c-8d23-11e9-87cd-0a0027000007,
> key: jms-session, data:
> 2019-06-12 08:07:25,173 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841002: created
> session name: c964d15d-8d23-11e9-87cd-0a0027000007, session connectionID:
> a77a2904
> 2019-06-12 08:07:25,178 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841004: added
> session metadata for session name : c964d15d-8d23-11e9-87cd-0a0027000007,
> key: jms-session, data:
>
> ...
>
> Here slave server logs:
>
> 2019-06-12 08:06:33,465 INFO
> [org.apache.activemq.artemis.integration.bootstrap] AMQ101000: Starting
> ActiveMQ Artemis Server
> 2019-06-12 08:06:33,534 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221000: backup Message Broker is starting with configuration Broker
> Configuration
>
> (clustered=true,journalDirectory=/vagrant_data/data/journal,bindingsDirectory=/vagrant_data/data/bindings,largeMessagesDirectory=/vagrant_data/data/large-messages,pagingDirectory=/vagrant_data/data/paging)
> 2019-06-12 08:06:33,561 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221032: Waiting to become backup node
> 2019-06-12 08:06:33,567 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221033: ** got backup lock
> 2019-06-12 08:06:33,596 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221054: libaio was found but the filesystem does not support AIO.
> Switching the configuration into NIO. Journal path:
> /vagrant_data/data/journal
> 2019-06-12 08:06:33,714 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221013: Using NIO Journal
> 2019-06-12 08:06:34,015 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221057: Global Max Size is being adjusted to 1/2 of the JVM max size
> (-Xmx). being defined as 1,073,741,824
> 2019-06-12 08:06:34,290 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221043: Protocol module found: [artemis-server]. Adding protocol support
> for: CORE
> 2019-06-12 08:06:34,295 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221043: Protocol module found: [artemis-amqp-protocol]. Adding protocol
> support for: AMQP
> 2019-06-12 08:06:34,306 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221043: Protocol module found: [artemis-hornetq-protocol]. Adding
> protocol support for: HORNETQ
> 2019-06-12 08:06:34,312 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221043: Protocol module found: [artemis-mqtt-protocol]. Adding protocol
> support for: MQTT
> 2019-06-12 08:06:34,319 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221043: Protocol module found: [artemis-openwire-protocol]. Adding
> protocol support for: OPENWIRE
> 2019-06-12 08:06:34,328 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221043: Protocol module found: [artemis-stomp-protocol]. Adding protocol
> support for: STOMP
> 2019-06-12 08:06:34,578 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221109: Apache ActiveMQ Artemis Backup Server version 2.9.0
> [46af3acc-7654-11e9-a97e-08002741bcc2] started, waiting live to fail before
> it gets active
> 2019-06-12 08:06:35,452 INFO
> [org.apache.activemq.hawtio.branding.PluginContextListener] Initialized
> activemq-branding plugin
> 2019-06-12 08:06:36,245 INFO
> [org.apache.activemq.hawtio.plugin.PluginContextListener] Initialized
> artemis-plugin plugin
> 2019-06-12 08:06:38,629 INFO  [io.hawt.HawtioContextListener] Initialising
> hawtio services
> 2019-06-12 08:06:38,951 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221080: Deploying address DLQ supporting [ANYCAST]
> 2019-06-12 08:06:38,952 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221003: Deploying ANYCAST queue DLQ on address DLQ
> 2019-06-12 08:06:38,953 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221080: Deploying address ExpiryQueue supporting [ANYCAST]
> 2019-06-12 08:06:38,954 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221003: Deploying ANYCAST queue ExpiryQueue on address ExpiryQueue
> 2019-06-12 08:06:39,058 INFO  [io.hawt.system.ConfigManager] Configuration
> will be discovered via system properties
> 2019-06-12 08:06:39,068 INFO  [io.hawt.jmx.JmxTreeWatcher] Welcome to
> hawtio
> 1.5.5 : http://hawt.io/ : Don't cha wish your console was hawt like me?
> ;-)
> 2019-06-12 08:06:39,118 INFO  [io.hawt.jmx.UploadManager] Using file upload
> directory: /var/lib/artemis/tmp/uploads
> 2019-06-12 08:06:39,261 INFO  [io.hawt.web.AuthenticationFilter] Starting
> hawtio authentication filter, JAAS realm: "activemq" authorized role(s):
> "amq" role principal classes:
> "org.apache.activemq.artemis.spi.core.security.jaas.RolePrincipal"
> 2019-06-12 08:06:39,356 INFO  [io.hawt.web.JolokiaConfiguredAgentServlet]
> Jolokia overridden property: [key=policyLocation,
> value=file:/var/lib/artemis/etc/jolokia-access.xml]
> 2019-06-12 08:06:39,441 INFO  [io.hawt.web.RBACMBeanInvoker] Using MBean
> [hawtio:type=security,area=jmx,rank=0,name=HawtioDummyJMXSecurity] for role
> based access control
> 2019-06-12 08:06:39,851 INFO  [io.hawt.system.ProxyWhitelist] Initial proxy
> whitelist: [localhost, 127.0.0.1, 10.0.2.15, 192.168.1.20, artemis-1S]
> 2019-06-12 08:06:39,959 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221020: Started EPOLL Acceptor at 0.0.0.0:61616 for protocols
> [CORE,MQTT,AMQP,HORNETQ,STOMP,OPENWIRE]
> 2019-06-12 08:06:39,960 INFO  [org.apache.activemq.artemis.core.server]
> AMQ221010: Backup Server is now live
> 2019-06-12 08:06:40,250 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841000: created
> connection: RemotingConnectionImpl [ID=e1b1f737, clientID=null,
> nodeID=46af3acc-7654-11e9-a97e-08002741bcc2,
>
> transportConnection=org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@1c515571
> [ID=e1b1f737,
> local= /192.168.1.20:61616, remote=/192.168.1.10:54368]]
> 2019-06-12 08:06:40,250 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841000: created
> connection: RemotingConnectionImpl [ID=a0e1f8ac, clientID=null,
> nodeID=46af3acc-7654-11e9-a97e-08002741bcc2,
>
> transportConnection=org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@1f2afd93
> [ID=a0e1f8ac,
> local= /192.168.1.20:61616, remote=/192.168.1.10:54370]]
> 2019-06-12 08:06:40,481 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP
> Server started at http://192.168.1.20:8161
> 2019-06-12 08:06:40,481 INFO  [org.apache.activemq.artemis] AMQ241002:
> Artemis Jolokia REST API available at
> http://192.168.1.20:8161/console/jolokia
> 2019-06-12 08:06:40,481 INFO  [org.apache.activemq.artemis] AMQ241004:
> Artemis Console available at http://192.168.1.20:8161/console
> 2019-06-12 08:06:54,451 INFO  [io.hawt.web.LoginServlet] hawtio login is
> using 1800 sec. HttpSession timeout
> 2019-06-12 08:08:39,951 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841001: destroyed
> connection: RemotingConnectionImpl [ID=e1b1f737, clientID=null,
> nodeID=46af3acc-7654-11e9-a97e-08002741bcc2,
>
> transportConnection=org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@1c515571
> [ID=e1b1f737,
> local= /192.168.1.20:61616, remote=/192.168.1.10:54368]]
> 2019-06-12 08:08:39,961 INFO
> [org.apache.activemq.artemis.core.server.plugin.impl] AMQ841001: destroyed
> connection: RemotingConnectionImpl [ID=a0e1f8ac, clientID=null,
> nodeID=46af3acc-7654-11e9-a97e-08002741bcc2,
>
> transportConnection=org.apache.activemq.artemis.core.remoting.impl.netty.NettyServerConnection@1f2afd93
> [ID=a0e1f8ac,
> local= /192.168.1.20:61616, remote=/192.168.1.10:54370]]
> 2019-06-12 08:10:35,979 WARN  [org.apache.activemq.artemis.core.client]
> AMQ212037: Connection failure has been detected: AMQ219011: Did not receive
> data from server for
>
> org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnection@f6a0824
> [ID=b80942d6,
> local= /192.168.1.20:40612, remote=artemis-1M/192.168.1.10:61616]
> [code=CONNECTION_TIMEDOUT]
>
>
>
>
>
> --
> Sent from:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message