activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mmp <m....@gmx.at>
Subject SQL Exception after failover using shared filesystem master/slave
Date Thu, 27 Mar 2008 10:12:48 GMT

Hi,

We are having two boxes, with 4 brokers each, which are responsible for one
queue. Failover works most of the time, but sometimes after a broker shuts
down, the one, which is waiting for the lock throws following exception:

java.io.IOException: Failed to start database 'derbydb', see the next
exception for details.
        at
org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:45)
        at
org.apache.activemq.store.jdbc.TransactionContext.getConnection(TransactionContext.java:61)
        at
org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.createAdapter(JDBCPersistenceAdapter.java:294)
        at
org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.getAdapter(JDBCPersistenceAdapter.java:249)
        at
org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.start(JDBCPersistenceAdapter.java:155)
        at
org.apache.activemq.store.journal.JournalPersistenceAdapter.start(JournalPersistenceAdapter.java:226)
        at
org.apache.activemq.broker.BrokerService.createRegionBroker(BrokerService.java:1549)
        at
org.apache.activemq.broker.BrokerService.createBroker(BrokerService.java:1502)
        at
org.apache.activemq.broker.BrokerService.getBroker(BrokerService.java:531)
        at
org.apache.activemq.broker.BrokerService.start(BrokerService.java:440)
        at
org.apache.activemq.xbean.XBeanBrokerService.afterPropertiesSet(XBeanBrokerService.java:46)
        at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:128
)
        at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1257)
        at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:438)
        at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:383)
        at java.security.AccessController.doPrivileged(Native Method)
        at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:353)
        at
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:245)
        at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:169)
        at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:242)
        at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
        at
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:400)
        at
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:736)
        at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:369)
        at
org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:64)
        at
org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:52)
        at
org.apache.activemq.xbean.XBeanBrokerFactory.createApplicationContext(XBeanBrokerFactory.java:91)
        at
org.apache.activemq.xbean.XBeanBrokerFactory.createBroker(XBeanBrokerFactory.java:51)
        at
org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:71)
        at
org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:54)
        at
org.apache.activemq.console.command.StartCommand.startBroker(StartCommand.java:112)
        at
org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:74)
        at
org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:51)
        at
org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:104)
        at
org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:51)
        at
org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:76)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.activemq.console.Main.runTaskClass(Main.java:222)
        at org.apache.activemq.console.Main.main(Main.java:106)
Caused by: SQL Exception: Failed to start database 'derbydb', see the next
exception for details.
        at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown
Source)
        at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown
Source)
        at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
Source)
        at
org.apache.derby.impl.jdbc.EmbedConnection.newSQLException(Unknown Source)
        at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown
Source)
        at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
        at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(Unknown
Source)
        at org.apache.derby.jdbc.Driver30.getNewEmbedConnection(Unknown
Source)
        at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
        at org.apache.derby.jdbc.EmbeddedDataSource.getConnection(Unknown
Source)
        at org.apache.derby.jdbc.EmbeddedDataSource.getConnection(Unknown
Source)
        at
org.apache.activemq.store.jdbc.TransactionContext.getConnection(TransactionContext.java:54)
        ... 40 more
ERROR: java.lang.RuntimeException: Failed to execute start task. Reason:
org.springframework.beans.factory.BeanCreationException: Error creating bean
with name 'org.apache.activemq.xbean.XBeanBrokerService#0' defined in class
path resource [activemq.xml]: Invocation of init method failed; nested ex
eption is java.io.IOException: Failed to start database 'derbydb', see the
next exception for details.
java.lang.RuntimeException: Failed to execute start task. Reason:
org.springframework.beans.factory.BeanCreationException: Error creating bean
with n
me 'org.apache.activemq.xbean.XBeanBrokerService#0' defined in class path
resource [activemq.xml]: Invocation of init method failed; nested exception
is java.io.IOException: Failed to start database 'derbydb', see the next
exception for details.
        at
org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:99)
        at
org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:51)
        at
org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:104)
        at
org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:51)
        at
org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:76)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.activemq.console.Main.runTaskClass(Main.java:222)
        at org.apache.activemq.console.Main.main(Main.java:106)
Caused by: org.springframework.beans.factory.BeanCreationException: Error
creating bean with name 'org.apache.activemq.xbean.XBeanBrokerService#0' de
ined in class path resource [activemq.xml]: Invocation of init method
failed; nested exception is java.io.IOException: Failed to start database
'derb
db', see the next exception for details.
        at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1260)
        at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:438)
        at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:383)
        at java.security.AccessController.doPrivileged(Native Method)
        at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:353)
        at
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:245)
        at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:169)
        at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:242)
        at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
        at
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:400)
        at
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:736)
        at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:369)
        at
org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:64)
        at
org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:52)
        at
org.apache.activemq.xbean.XBeanBrokerFactory.createApplicationContext(XBeanBrokerFactory.java:91)
        at
org.apache.activemq.xbean.XBeanBrokerFactory.createBroker(XBeanBrokerFactory.java:51)
        at
org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:71)
        at
org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:54)
        at
org.apache.activemq.console.command.StartCommand.startBroker(StartCommand.java:112)
        at
org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:74)
        ... 10 more
Caused by: java.io.IOException: Failed to start database 'derbydb', see the
next exception for details.
        at
org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:45)
        at
org.apache.activemq.store.jdbc.TransactionContext.getConnection(TransactionContext.java:61)
        at
org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.createAdapter(JDBCPersistenceAdapter.java:294)
        at
org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.getAdapter(JDBCPersistenceAdapter.java:249)
        at
org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.start(JDBCPersistenceAdapter.java:155)
        at
org.apache.activemq.store.journal.JournalPersistenceAdapter.start(JournalPersistenceAdapter.java:226)
        at
org.apache.activemq.broker.BrokerService.createRegionBroker(BrokerService.java:1549)
        at
org.apache.activemq.broker.BrokerService.createBroker(BrokerService.java:1502)
        at
org.apache.activemq.broker.BrokerService.getBroker(BrokerService.java:531)
        at
org.apache.activemq.broker.BrokerService.start(BrokerService.java:440)
        at
org.apache.activemq.xbean.XBeanBrokerService.afterPropertiesSet(XBeanBrokerService.java:46)
        at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:128
)
        at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1257)
        ... 29 more
Caused by: SQL Exception: Failed to start database 'derbydb', see the next
exception for details.
        at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown
Source)
        at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown
Source)
        at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
Source)
        at
org.apache.derby.impl.jdbc.EmbedConnection.newSQLException(Unknown Source)
        at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown
Source)
        at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
        at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(Unknown
Source)
        at org.apache.derby.jdbc.Driver30.getNewEmbedConnection(Unknown
Source)
        at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
        at org.apache.derby.jdbc.EmbeddedDataSource.getConnection(Unknown
Source)
        at org.apache.derby.jdbc.EmbeddedDataSource.getConnection(Unknown
Source)
        at
org.apache.activemq.store.jdbc.TransactionContext.getConnection(TransactionContext.java:54)
        ... 40 more
ERROR: java.lang.Exception:
org.springframework.beans.factory.BeanCreationException: Error creating bean
with name 'org.apache.activemq.xbean.XBeanBr
kerService#0' defined in class path resource [activemq.xml]: Invocation of
init method failed; nested exception is java.io.IOException: Failed to sta
t database 'derbydb', see the next exception for details.
java.lang.Exception:
org.springframework.beans.factory.BeanCreationException: Error creating bean
with name 'org.apache.activemq.xbean.XBeanBrokerSer
ice#0' defined in class path resource [activemq.xml]: Invocation of init
method failed; nested exception is java.io.IOException: Failed to start data
ase 'derbydb', see the next exception for details.
        at
org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:100)
        at
org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:51)
        at
org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:104)
        at
org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:51)
        at
org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:76)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.activemq.console.Main.runTaskClass(Main.java:222)
        at org.apache.activemq.console.Main.main(Main.java:106)
Caused by: org.springframework.beans.factory.BeanCreationException: Error
creating bean with name 'org.apache.activemq.xbean.XBeanBrokerService#0' de
ined in class path resource [activemq.xml]: Invocation of init method
failed; nested exception is java.io.IOException: Failed to start database
'derb
db', see the next exception for details.
        at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1260)
        at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:438)
        at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory$1.run(AbstractAutowireCapableBeanFactory.java:383)
        at java.security.AccessController.doPrivileged(Native Method)
        at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:353)
        at
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:245)
        at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:169)
        at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:242)
        at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:164)
        at
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:400)
        at
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:736)
        at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:369)
        at
org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:64)
        at
org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:52)
        at
org.apache.activemq.xbean.XBeanBrokerFactory.createApplicationContext(XBeanBrokerFactory.java:91)
        at
org.apache.activemq.xbean.XBeanBrokerFactory.createBroker(XBeanBrokerFactory.java:51)
        at
org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:71)
        at
org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:54)
        at
org.apache.activemq.console.command.StartCommand.startBroker(StartCommand.java:112)
        at
org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:74)
        ... 10 more
Caused by: java.io.IOException: Failed to start database 'derbydb', see the
next exception for details.
        at
org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:45)
        at
org.apache.activemq.store.jdbc.TransactionContext.getConnection(TransactionContext.java:61)
        at
org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.createAdapter(JDBCPersistenceAdapter.java:294)
        at
org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.getAdapter(JDBCPersistenceAdapter.java:249)
        at
org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.start(JDBCPersistenceAdapter.java:155)
        at
org.apache.activemq.store.journal.JournalPersistenceAdapter.start(JournalPersistenceAdapter.java:226)
        at
org.apache.activemq.broker.BrokerService.createRegionBroker(BrokerService.java:1549)
        at
org.apache.activemq.broker.BrokerService.createBroker(BrokerService.java:1502)
        at
org.apache.activemq.broker.BrokerService.getBroker(BrokerService.java:531)
        at
org.apache.activemq.broker.BrokerService.start(BrokerService.java:440)
        at
org.apache.activemq.xbean.XBeanBrokerService.afterPropertiesSet(XBeanBrokerService.java:46)
        at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:128
)
        at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1257)
        ... 29 more
Caused by: SQL Exception: Failed to start database 'derbydb', see the next
exception for details.
        at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown
Source)
        at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown
Source)
        at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
Source)
        at
org.apache.derby.impl.jdbc.EmbedConnection.newSQLException(Unknown Source)
        at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown
Source)
        at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
        at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(Unknown
Source)
        at org.apache.derby.jdbc.Driver30.getNewEmbedConnection(Unknown
Source)
        at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
        at org.apache.derby.jdbc.EmbeddedDataSource.getConnection(Unknown
Source)
        at org.apache.derby.jdbc.EmbeddedDataSource.getConnection(Unknown
Source)
        at
org.apache.activemq.store.jdbc.TransactionContext.getConnection(TransactionContext.java:54)
        ... 40 more

After that, clients can not establish a connection and the newly started
broker, which throws the exception is holding the lock. Restarting the
original broker, also doesn't get the lock. It worked fine, after sending
about 34 000 000 messages over the night, having two failovers (which
worked).

This is the config of one the brokers:

  <broker xmlns="http://activemq.org/config/1.0"
brokerName="localhost-My-QUEUE" persistent="false" useJmx="true">
    <!-- The transport connectors ActiveMQ will listen to -->
    <transportConnectors>
       <transportConnector name="tc-localhost-My-QUEUE"
uri="tcp://localhost:61616"/>
    </transportConnectors>

    <persistenceAdapter>
   	<journaledJDBC dataDirectory="\\192.0.0.2\My-QUEUE"/>
    </persistenceAdapter>

    <managementContext>
	<managementContext connectorPort="2011"/>
    </managementContext>
  </broker>


We are using a Windows XP SP2 Share with NTFS.

thx in advance for any help



-- 
View this message in context: http://www.nabble.com/SQL-Exception-after-failover-using-shared-filesystem-master-slave-tp16323606s2354p16323606.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Mime
View raw message