activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandro Tosi <sandro.t...@gmail.com>
Subject Re: Persistance on DB and "Failed to checkpoint a message store" error
Date Mon, 22 Sep 2008 11:54:15 GMT

Really no-one knows how to fix this big problem? this is a major stop for
activemq adoption in our organization.

I appreciate any input on this.

Thanks in advance,
Sandro


Sandro Tosi wrote:
> 
> Sorry for missing this in the initial post, here it is:
> 
> # ./bin/activemq --version
> ACTIVEMQ_HOME: /usr/local/apache-activemq-5.1.0
> ACTIVEMQ_BASE: /usr/local/apache-activemq-5.1.0
> 
> ActiveMQ 5.1.0
> For help or more information please see: http://activemq.apache.org
> 
> Thanks,
> Sandro
> 
> 
> rajdavies wrote:
>> 
>> which version of activemq ?
>> 
>> On 18 Sep 2008, at 07:51, Sandro Tosi wrote:
>> 
>>>
>>> Hello,
>>> my filesystem filled up due to ActiveMQ keep writing into nohup.out  
>>> this
>>> error message:
>>>
>>> ERROR JournalPersistenceAdapter      - Failed to checkpoint a  
>>> message store:
>>> java.util.concurrent.ExecutionException: java.io.IOException: Already
>>> started.
>>> java.util.concurrent.ExecutionException: java.io.IOException: Already
>>> started.
>>>        at
>>> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:205)
>>>        at java.util.concurrent.FutureTask.get(FutureTask.java:80)
>>>        at
>>> org 
>>> .apache 
>>> .activemq 
>>> .store 
>>> .journal 
>>> .JournalPersistenceAdapter 
>>> .doCheckpoint(JournalPersistenceAdapter.java:398)
>>>        at
>>> org.apache.activemq.store.journal.JournalPersistenceAdapter 
>>> $1.iterate(JournalPersistenceAdapter.java:119)
>>>        at
>>> org 
>>> .apache 
>>> .activemq 
>>> .thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:98)
>>>        at
>>> org.apache.activemq.thread.DedicatedTaskRunner 
>>> $1.run(DedicatedTaskRunner.java:36)
>>> Caused by: java.io.IOException: Already started.
>>>        at
>>> org 
>>> .apache 
>>> .activemq 
>>> .store.jdbc.TransactionContext.begin(TransactionContext.java:148)
>>>        at
>>> org 
>>> .apache 
>>> .activemq 
>>> .store 
>>> .jdbc 
>>> .JDBCPersistenceAdapter.beginTransaction(JDBCPersistenceAdapter.java: 
>>> 356)
>>>        at
>>> org 
>>> .apache 
>>> .activemq 
>>> .store 
>>> .journal 
>>> .JournalPersistenceAdapter 
>>> .beginTransaction(JournalPersistenceAdapter.java:193)
>>>        at
>>> org 
>>> .apache 
>>> .activemq.util.TransactionTemplate.run(TransactionTemplate.java:41)
>>>        at
>>> org 
>>> .apache 
>>> .activemq 
>>> .store 
>>> .journal.JournalMessageStore.checkpoint(JournalMessageStore.java:258)
>>>        at
>>> org 
>>> .apache 
>>> .activemq 
>>> .store 
>>> .journal.JournalMessageStore.checkpoint(JournalMessageStore.java:233)
>>>        at
>>> org.apache.activemq.store.journal.JournalPersistenceAdapter 
>>> $4.call(JournalPersistenceAdapter.java:368)
>>>        at
>>> org.apache.activemq.store.journal.JournalPersistenceAdapter 
>>> $4.call(JournalPersistenceAdapter.java:367)
>>>        at
>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
>>>        at java.util.concurrent.FutureTask.run(FutureTask.java:123)
>>>        at
>>> java.util.concurrent.ThreadPoolExecutor 
>>> $Worker.runTask(ThreadPoolExecutor.java:650)
>>>        at
>>> java.util.concurrent.ThreadPoolExecutor 
>>> $Worker.run(ThreadPoolExecutor.java:675)
>>>        at java.lang.Thread.run(Thread.java:595)
>>>
>>> This is happening since our database went offline due to backup (last
>>> night). Indeed, when I delete a msg from the web admin console, it's  
>>> not
>>> purged on the db because in the log I can see:
>>>
>>> ERROR DefaultDatabaseLocker          - Failed to update database lock:
>>> java.sql.SQLException: Closed Connection
>>> java.sql.SQLException: Closed Connection
>>>        at
>>> oracle 
>>> .jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:125)
>>>        at
>>> oracle 
>>> .jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:162)
>>>        at
>>> oracle 
>>> .jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:227)
>>>        at
>>> oracle.jdbc.driver.OracleStatement.ensureOpen(OracleStatement.java: 
>>> 3291)
>>>        at
>>> oracle 
>>> .jdbc 
>>> .driver 
>>> .OraclePreparedStatement 
>>> .executeInternal(OraclePreparedStatement.java:2966)
>>>        at
>>> oracle 
>>> .jdbc 
>>> .driver 
>>> .OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java: 
>>> 3057)
>>>        at
>>> org 
>>> .apache 
>>> .commons 
>>> .dbcp 
>>> .DelegatingPreparedStatement 
>>> .executeUpdate(DelegatingPreparedStatement.java:94)
>>>        at
>>> org 
>>> .apache 
>>> .commons 
>>> .dbcp 
>>> .DelegatingPreparedStatement 
>>> .executeUpdate(DelegatingPreparedStatement.java:94)
>>>        at
>>> org 
>>> .apache 
>>> .activemq 
>>> .store 
>>> .jdbc.DefaultDatabaseLocker.keepAlive(DefaultDatabaseLocker.java:104)
>>>        at
>>> org 
>>> .apache 
>>> .activemq 
>>> .store 
>>> .jdbc 
>>> .JDBCPersistenceAdapter 
>>> .databaseLockKeepAlive(JDBCPersistenceAdapter.java:458)
>>>        at
>>> org.apache.activemq.store.jdbc.JDBCPersistenceAdapter 
>>> $3.run(JDBCPersistenceAdapter.java:260)
>>>        at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java: 
>>> 417)
>>>        at
>>> java.util.concurrent.FutureTask 
>>> $Sync.innerRunAndReset(FutureTask.java:280)
>>>        at  
>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:135)
>>>        at
>>> java.util.concurrent.ScheduledThreadPoolExecutor 
>>> $ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:65)
>>>        at
>>> java.util.concurrent.ScheduledThreadPoolExecutor 
>>> $ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:142)
>>>        at
>>> java.util.concurrent.ScheduledThreadPoolExecutor 
>>> $ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:166)
>>>        at
>>> java.util.concurrent.ThreadPoolExecutor 
>>> $Worker.runTask(ThreadPoolExecutor.java:650)
>>>        at
>>> java.util.concurrent.ThreadPoolExecutor 
>>> $Worker.run(ThreadPoolExecutor.java:675)
>>>        at java.lang.Thread.run(Thread.java:595)
>>> INFO  JDBCPersistenceAdapter         - No longer able to keep the  
>>> exclusive
>>> lock so giving up being a master
>>> WARN  JDBCPersistenceAdapter         - Failed to stop broker
>>>
>>> Is there something we can do to avoid such situation? To auto- 
>>> reconnect to
>>> (Oracle) db after an offline backup? We are piloting to introduce  
>>> activemq
>>> in our production env, and this issue is scaring us a bit.
>>>
>>> If you need, I can provide the xml config file and any other info  
>>> required.
>>>
>>> Thanks in advance,
>>> Sandro
>>> -- 
>>> View this message in context:
>>> http://www.nabble.com/Persistance-on-DB-and-%22Failed-to-checkpoint-a-message-store%22-error-tp19547006p19547006.html
>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>
>> 
>> 
>> 
> 
> 

-- 
View this message in context: http://www.nabble.com/Persistance-on-DB-and-%22Failed-to-checkpoint-a-message-store%22-error-tp19547006p19606482.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Mime
View raw message