activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dirk Fröhner <dirk.froeh...@email.de>
Subject Re: Master/Slave Failover with JDBC blocks Tomcat startup
Date Tue, 04 Jan 2011 15:24:39 GMT
Albrecht,

to infinitely try to obtain the lock on the shared persistence layer is exactly a slave's
job. It does nothing else before that happens. The process that has the lock automatically
is the master, the process that waits for the lock automatically is the slave. This applies
to all additional processes that you put in place for high availability (although it is pure
paranoia to put more than two slaves into the game). A master/slave architecture is only meaningful
when all involved processes reside on dedicated hosts. When you have all processes running
on one machine you'll lose everything when that machine dies and it is a broken architecture.
A master/slave architecture is used for high availability of _one_ JMS node (as we call it
here). It is a concept that is vertical to horizontal scalability.

Excuse me for not looking into your Spring config, it is so unreadable... therefore I only
answered in general to your text... :-)

Glück auf!
Dirk


On Jan 4, 2011, at 11:35 AM, Albrecht Militzer wrote:

> 
> I have MySQL 5.1, I have InnoDB-Tables, but Slaves do not start up. The first
> process becomes master. The others block while trying to become master. They
> never become slaves. They try to execute 
> 
> SELECT * FROM ACTIVEMQ_LOCK FOR UPDATE. 
> 
> This waits for the lock to be available and finally ends with a timeout. The
> next line in the log is: 
> 
> Failed to acquire lock. Sleeping for 1000 milli(s) before trying again...
> 
> This keeps going forever. The server shows the line "Started Jetty Server"
> and accepts HTTP requests. The process that is to become slave never does
> either of this.
> 
> 1) If I switch to Derby, the problem goes away.
> 
> 2) In MySQL, I had some luck with changing the lock statement to 
> 
> SELECT * FROM ACTIVEMQ_LOCK LOCK IN SHARE MODE
> 
> Some luck only because:
> 
> 1) Failover does not work. Once I kill the master, no slave becomes master.
> 2) Even this is not reliable. I cannot reproduce it reliably, but sometime,
> the slave still does not start up.
> 
> Here is my Spring app-context:
> 
> <?xml version="1.0" encoding="UTF-8"?>
> <beans xmlns="http://www.springframework.org/schema/beans"
> 	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xmlns:tx="http://www.springframework.org/schema/tx"
> 	xmlns:context="http://www.springframework.org/schema/context"
> xmlns:p="http://www.springframework.org/schema/p"
> 	xmlns:jms="http://www.springframework.org/schema/jms"
> xmlns:amq="http://activemq.apache.org/schema/core"
> 	xsi:schemaLocation="
>    http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
>    http://www.springframework.org/schema/tx
> http://www.springframework.org/schema/tx/spring-tx-3.0.xsd
>    http://www.springframework.org/schema/context
> http://www.springframework.org/schema/context/spring-context-3.0.xsd
>    http://www.springframework.org/schema/jms
> http://www.springframework.org/schema/jms/spring-jms.xsd
> 	http://activemq.apache.org/schema/core
> http://activemq.apache.org/schema/core/activemq-core-5.4.2.xsd">
> 
> 	<context:annotation-config />
> 	<context:component-scan base-package="de.jeha" />
> 
> 	<bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource"
> 		destroy-method="close">
> 		<property name="driverClass" value="com.mysql.jdbc.Driver" />
> 		<property name="jdbcUrl"
> 		
> value="jdbc:mysql://${de.jeha.db.host}:3306/test?autoReconnectForPools=true"
> />
> 		<property name="user" value="amqtest" />
> 		<property name="password" value="amqtest" />
> 		<property name="initialPoolSize" value="1" />
> 		<property name="minPoolSize" value="1" />
> 		<property name="maxPoolSize" value="3" />
> 		<property name="preferredTestQuery" value="SELECT 1;" />
> 		<property name="idleConnectionTestPeriod" value="300" />
> 	</bean>
> 
> 	<bean id="entityManagerFactory"
> 		class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean"
> 		p:dataSource-ref="dataSource" />
> 
> 	<bean id="transactionManager"
> class="org.springframework.orm.jpa.JpaTransactionManager"
> 		p:entityManagerFactory-ref="entityManagerFactory" />
> 
> 	<bean id="mysql-ds" class="org.apache.commons.dbcp.BasicDataSource"
> 		destroy-method="close">
>        <property name="driverClassName" value="com.mysql.jdbc.Driver" />
> 		<property name="url"
> 		
> value="jdbc:mysql://${de.jeha.db.host}:3306/test_activemq?relaxAutoCommit=true&amp;profileSQL=true"
> /> <!-- &amp;profileSQL=true -->
> 
>        <property name="username" value="amqtest" />
>        <property name="password" value="amqtest" />
> 
> 
> 		<property name="poolPreparedStatements" value="true" />
> 	</bean>
> 
> 	<bean id="derby-ds" class="org.apache.commons.dbcp.BasicDataSource"
> 		destroy-method="close">
> 		<property name="driverClassName"
> value="org.apache.derby.jdbc.ClientDriver" />
> 		<property name="url" value="jdbc:derby://localhost:1527/amqtest" />
> 
>        <property name="username" value="default" />
>        <property name="password" value="default" />
> 
> 		<property name="poolPreparedStatements" value="true" />
> 	</bean>
> 
> 	<amq:broker id="broker" useJmx="true" brokerName="brokerName1">
> 		<amq:managementContext>
> 			<amq:managementContext connectorPort="${de.jeha.jmxport}"
> jmxDomainName="org.apache.activemq" />
> 		</amq:managementContext>
> 		<amq:persistenceAdapter>
> 			<amq:jdbcPersistenceAdapter dataSource="#mysql-ds" useDatabaseLock="true"
> createTablesOnStartup="true">
> 			<!--<amq:jdbcPersistenceAdapter dataSource="#derby-ds"
> useDatabaseLock="true" createTablesOnStartup="true">-->
> 				<amq:statements>
> 
>                    <amq:statements lockTableName="ACTIVEMQ_LOCK"
>                                    lockCreateStatement="SELECT * FROM
> ACTIVEMQ_LOCK LOCK IN SHARE MODE"
>                                    messageTableName="ACTIVEMQ_MSGS"
> durableSubAcksTableName="ACTIVEMQ_ACKS"/>
>                </amq:statements>
> 			</amq:jdbcPersistenceAdapter>
> 		</amq:persistenceAdapter>
> 		<amq:plugins>
> 			<amq:statisticsBrokerPlugin />
> 		</amq:plugins>
> 		<amq:transportConnectors>
> 			<!--<amq:transportConnector name="default"
> uri="tcp://10.121.123.26:61616" />-->
> 			<amq:transportConnector name="default"
> uri="tcp://10.121.123.26:${de.jeha.amq.host.port}" />
> 		</amq:transportConnectors>
> 	</amq:broker>
> 
> 	<amq:queue id="destination" physicalName="amq.test" />
> 
> 	<amq:connectionFactory id="jmsFactory"
> 	
> brokerURL="failover:(tcp://10.121.123.26:61616,tcp://10.121.123.26:62616)?initialReconnectDelay=100"
> />
> 
> 	<bean id="jmsTransactionManager"
> 		class="org.springframework.jms.connection.JmsTransactionManager"
> 		p:connectionFactory-ref="jmsFactory" />
> 
> 	<bean id="jmsConnectionFactory"
> 		class="org.springframework.jms.connection.SingleConnectionFactory"
> 		p:targetConnectionFactory-ref="jmsFactory" />
> 
> 	<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate"
> 		p:connectionFactory-ref="jmsConnectionFactory"
> 		p:defaultDestination-ref="destination" />
> 
> 	<bean id="simpleJMSReceiver"
> 		class="de.jeha.spring_activemq_webapp.struts2.mdp.SimpleJMSReceiver" />
> 
> 	<bean id="jmsMessageListener"
> 		class="org.springframework.jms.listener.adapter.MessageListenerAdapter">
> 		<constructor-arg ref="simpleJMSReceiver" />
> 		<property name="messageConverter">
> 			<null />
> 		</property>
> 	</bean>
> 
> 	<jms:listener-container container-type="default"
> 		connection-factory="jmsFactory" acknowledge="auto">
> 		<jms:listener destination="amq.test" ref="jmsMessageListener" />
> 	</jms:listener-container>
> 
> </beans>
> 
> I start my test processes with different values for $de.jeha.amq.host.port.
> The values are those listed in the failover url:
> failover:(tcp://10.121.123.26:61616,tcp://10.121.123.26:62616)
> 
> 
> Thanks in advance for anything that could point me in the right direction.
> 
> Albrecht
> -- 
> View this message in context: http://activemq.2283324.n4.nabble.com/Master-Slave-Failover-with-JDBC-blocks-Tomcat-startup-tp3173318p3173318.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Mime
View raw message