db-derby-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Wu <...@terracottatech.com>
Subject Re: Corrupted Derby DB
Date Thu, 26 May 2011 23:22:51 GMT
Dag,

Thanks for the quick response. The db still fails to boot with the non-debug jar, so the consistency
check is unfortunately a no-go. So it's seeming more and more like a derby bug. Also double
checked the disk on the machine where this failed, seemed healthy.

I've filed DERBY-5248 and attached the db to it.

Thanks,
Tim Wu
On May 26, 2011, at 3:53 PM, Dag H. Wanvik wrote:

> 
> Hi Tim,
> 
> you could file a new JIRA issue and attach the database as a zip file if
> it's not too large.
> 
> http://db.apache.org/derby/DerbyBugGuidelines.html
> 
> I see you use the debug version of Derby. In this particular case, it
> would be interesting to know what happens if you run Derby with the
> non-debug version, since it would run without the assert check that hit
> you. If you are able to boot then (please save the database first), you
> should run a full consistency check on the database, see
> http://wiki.apache.org/db-derby/DatabaseConsistencyCheck.
> 
> Should this work out OK, it could mean that the assert was wrong. Maybe
> not very likely, I am not familiar with this code, but worth a try. If
> it crashes or fails the consistency checking it sounds like a Derby bug
> to me.
> 
> Thanks,
> Dag
> 
> 
> 
> Tim Wu <twu@terracottatech.com> writes:
> 
>> Hi,
>> 
>> During some crash tests for our product with embedded derby, we hit a
>> situation where after the crash, derby's db appears to have become
>> corrupted. There aren't logs as to what was going on at the exact
>> moment of the crash, but the db fails to boot with the following
>> error. This test was done using Derby 10.8.1.2, and we were unable to
>> reproduce the exact problem again. Seems like it could be a problem if
>> it were possible to cause Derby's DB to become corrupted just by
>> bringing down the process. Is this a Derby bug?
>> 
>> I do have the offending db zipped up, just not quite sure how to share
>> it with the mailing list. Let me know the preferred method, and I'll
>> send it across.
>> 
>> org.apache.derby.shared.common.sanity.AssertFailure: ASSERT FAILED
>> incorrect deleted row count. Should be: 903, instead got: 902, maxSlot
>> = 1255, recordCount = 1255
>> at
>> org.apache.derby.shared.common.sanity.SanityManager.THROWASSERT(SanityManager.java:162)
>> at
>> org.apache.derby.shared.common.sanity.SanityManager.THROWASSERT(SanityManager.java:147)
>> at
>> org.apache.derby.impl.store.raw.data.BasePage.internalNonDeletedRecordCount(BasePage.java:1432)
>> at org.apache.derby.impl.store.raw.data.CachedPage.releaseExclusive(CachedPage.java:488)

>> at org.apache.derby.impl.store.raw.data.StoredPage.releaseExclusive(StoredPage.java:1066)

>> at org.apache.derby.impl.store.raw.data.BasePage.unlatch(BasePage.java:1370) 
>> at
>> org.apache.derby.impl.store.raw.data.PageBasicOperation.releaseResource(PageBasicOperation.java:195)
>> at
>> org.apache.derby.impl.store.raw.data.PhysicalUndoOperation.releaseResource(PhysicalUndoOperation.java:177)
>> at org.apache.derby.impl.store.raw.log.FileLogger.undo(FileLogger.java:1055) 
>> at org.apache.derby.impl.store.raw.xact.Xact.abort(Xact.java:952) 
>> at
>> org.apache.derby.impl.store.raw.xact.XactFactory.rollbackAllTransactions(XactFactory.java:547)
>> at org.apache.derby.impl.store.raw.log.LogToFile.recover(LogToFile.java:1229) 
>> at org.apache.derby.impl.store.raw.RawStore.boot(RawStore.java:339) 
>> at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1982)

>> at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:334)

>> at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:531)

>> at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427)

>> at org.apache.derby.impl.store.access.RAMAccessManager.boot(RAMAccessManager.java:1019)

>> at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1982)

>> at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:334)

>> at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:531)

>> at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427)

>> at org.apache.derby.impl.db.BasicDatabase.bootStore(BasicDatabase.java:749) 
>> at org.apache.derby.impl.db.BasicDatabase.boot(BasicDatabase.java:177) 
>> at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1982)

>> at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:334)

>> at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(BaseMonitor.java:1816)

>> at
>> org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(BaseMonitor.java:1682)
>> at
>> org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(BaseMonitor.java:1560)
>> at
>> org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(BaseMonitor.java:979)
>> at
>> org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Monitor.java:550)
>> at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(EmbedConnection.java:2697)

>> at org.apache.derby.impl.jdbc.EmbedConnection.<init>(EmbedConnection.java:385)

>> at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(EmbedConnection30.java:73)

>> at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(EmbedConnection40.java:51)

>> at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Driver40.java:70) 
>> at org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:248) 
>> at org.apache.derby.jdbc.AutoloadedDriver.connect(AutoloadedDriver.java:144) 
>> at java.sql.DriverManager.getConnection(DriverManager.java:582) 
>> at java.sql.DriverManager.getConnection(DriverManager.java:154) 
>> at OpenDerby.main(OpenDerby.java:17) 
>> --------------- 
>> Stack traces for all live threads: 
>> Thread name=main id=1 priority=5 state=RUNNABLE isdaemon=false 
>> java.lang.Thread.dumpThreads(Native Method) 
>> java.lang.Thread.getAllStackTraces(Thread.java:1554) 
>> org.apache.derby.shared.common.sanity.ThreadDump.getStackDumpString(ThreadDump.java:34)

>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

>> java.lang.reflect.Method.invoke(Method.java:597) 
>> org.apache.derby.shared.common.sanity.AssertFailure$1.run(AssertFailure.java:165)

>> java.security.AccessController.doPrivileged(Native Method) 
>> org.apache.derby.shared.common.sanity.AssertFailure.dumpThreads(AssertFailure.java:159)

>> org.apache.derby.shared.common.sanity.AssertFailure.<init>(AssertFailure.java:72)

>> org.apache.derby.shared.common.sanity.SanityManager.THROWASSERT(SanityManager.java:162)

>> org.apache.derby.shared.common.sanity.SanityManager.THROWASSERT(SanityManager.java:147)

>> org.apache.derby.impl.store.raw.data.BasePage.internalNonDeletedRecordCount(BasePage.java:1432)
>> org.apache.derby.impl.store.raw.data.CachedPage.releaseExclusive(CachedPage.java:488)

>> org.apache.derby.impl.store.raw.data.StoredPage.releaseExclusive(StoredPage.java:1066)

>> org.apache.derby.impl.store.raw.data.BasePage.unlatch(BasePage.java:1370) 
>> org.apache.derby.impl.store.raw.data.PageBasicOperation.releaseResource(PageBasicOperation.java:195)
>> org.apache.derby.impl.store.raw.data.PhysicalUndoOperation.releaseResource(PhysicalUndoOperation.java:177)
>> org.apache.derby.impl.store.raw.log.FileLogger.undo(FileLogger.java:1055) 
>> org.apache.derby.impl.store.raw.xact.Xact.abort(Xact.java:952) 
>> org.apache.derby.impl.store.raw.xact.XactFactory.rollbackAllTransactions(XactFactory.java:547)
>> org.apache.derby.impl.store.raw.log.LogToFile.recover(LogToFile.java:1229) 
>> org.apache.derby.impl.store.raw.RawStore.boot(RawStore.java:339) 
>> org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1982) 
>> org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:334)

>> org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:531)

>> org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427)

>> org.apache.derby.impl.store.access.RAMAccessManager.boot(RAMAccessManager.java:1019)

>> org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1982) 
>> org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:334)

>> org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:531)

>> org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427)

>> org.apache.derby.impl.db.BasicDatabase.bootStore(BasicDatabase.java:749) 
>> org.apache.derby.impl.db.BasicDatabase.boot(BasicDatabase.java:177) 
>> org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1982) 
>> org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:334)

>> org.apache.derby.impl.services.monitor.BaseMonitor.bootService(BaseMonitor.java:1816)

>> org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(BaseMonitor.java:1682)
>> org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(BaseMonitor.java:1560)
>> org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(BaseMonitor.java:979)
>> org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Monitor.java:550)

>> org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(EmbedConnection.java:2697)

>> org.apache.derby.impl.jdbc.EmbedConnection.<init>(EmbedConnection.java:385)

>> org.apache.derby.impl.jdbc.EmbedConnection30.<init>(EmbedConnection30.java:73)

>> org.apache.derby.impl.jdbc.EmbedConnection40.<init>(EmbedConnection40.java:51)

>> org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Driver40.java:70) 
>> org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:248) 
>> org.apache.derby.jdbc.AutoloadedDriver.connect(AutoloadedDriver.java:144) 
>> java.sql.DriverManager.getConnection(DriverManager.java:582) 
>> java.sql.DriverManager.getConnection(DriverManager.java:154) 
>> OpenDerby.main(OpenDerby.java:17) 
>> 
>> Thread name=derby.rawStoreDaemon id=12 priority=5 state=TIMED_WAITING isdaemon=true

>> java.lang.Object.wait(Native Method) 
>> org.apache.derby.impl.services.daemon.BasicDaemon.rest(BasicDaemon.java:576) 
>> org.apache.derby.impl.services.daemon.BasicDaemon.run(BasicDaemon.java:390) 
>> java.lang.Thread.run(Thread.java:680) 
>> 
>> Thread name=Finalizer id=3 priority=8 state=WAITING isdaemon=true 
>> java.lang.Object.wait(Native Method) 
>> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118) 
>> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134) 
>> java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159) 
>> 
>> Thread name=Reference Handler id=2 priority=10 state=WAITING isdaemon=true 
>> java.lang.Object.wait(Native Method) 
>> java.lang.Object.wait(Object.java:485) 
>> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116) 
>> 
>> Thread name=Signal Dispatcher id=6 priority=9 state=RUNNABLE isdaemon=true 
>> 
>> Thread name=Timer-0 id=10 priority=5 state=WAITING isdaemon=true 
>> java.lang.Object.wait(Native Method) 
>> java.lang.Object.wait(Object.java:485) 
>> java.util.TimerThread.mainLoop(Timer.java:483) 
>> java.util.TimerThread.run(Timer.java:462) 
>> 
>> 
>> --------------- 
>> 
>> Exception in thread "main" java.sql.SQLException: Failed to start
>> database
>> /Users/tim/Downloads/LinkedBlockingQueueCrashTest/objectdb/datadb'
>> with class loader sun.misc.Launcher$AppClassLoader@40affc70, see the
>> next exception for details.
>> at
>> org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:98)
>> at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Util.java:142) 
>> at org.apache.derby.impl.jdbc.Util.seeNextException(Util.java:278) 
>> at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(EmbedConnection.java:2736)

>> at org.apache.derby.impl.jdbc.EmbedConnection.<init>(EmbedConnection.java:385)

>> at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(EmbedConnection30.java:73)

>> at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(EmbedConnection40.java:51)

>> at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Driver40.java:70) 
>> at org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:248) 
>> at org.apache.derby.jdbc.AutoloadedDriver.connect(AutoloadedDriver.java:144) 
>> at java.sql.DriverManager.getConnection(DriverManager.java:582) 
>> at java.sql.DriverManager.getConnection(DriverManager.java:154) 
>> at OpenDerby.main(OpenDerby.java:17) 
>> Caused by: java.sql.SQLException: Failed to start database
>> /Users/tim/Downloads/LinkedBlockingQueueCrashTest/objectdb/datadb'
>> with class loader sun.misc.Launcher$AppClassLoader@40affc70, see the
>> next exception for details.
>> at
>> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(SQLExceptionFactory.java:45)
>> at
>> org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(SQLExceptionFactory40.java:122)
>> at
>> org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:71)
>> ... 12 more 
>> Caused by: java.sql.SQLException: Java exception: 'ASSERT FAILED
>> incorrect deleted row count. Should be: 903, instead got: 902, maxSlot
>> = 1255, recordCount = 1255:
>> org.apache.derby.shared.common.sanity.AssertFailure'.
>> at
>> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(SQLExceptionFactory.java:45)
>> at
>> org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(SQLExceptionFactory40.java:122)
>> at
>> org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:71)
>> at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Util.java:142) 
>> at org.apache.derby.impl.jdbc.Util.javaException(Util.java:299) 
>> at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(EmbedConnection.java:2732)

>> ... 9 more 
>> Caused by: org.apache.derby.shared.common.sanity.AssertFailure: ASSERT
>> FAILED incorrect deleted row count. Should be: 903, instead got: 902,
>> maxSlot = 1255, recordCount = 1255
>> at
>> org.apache.derby.shared.common.sanity.SanityManager.THROWASSERT(SanityManager.java:162)
>> at
>> org.apache.derby.shared.common.sanity.SanityManager.THROWASSERT(SanityManager.java:147)
>> at
>> org.apache.derby.impl.store.raw.data.BasePage.internalNonDeletedRecordCount(BasePage.java:1432)
>> at org.apache.derby.impl.store.raw.data.CachedPage.releaseExclusive(CachedPage.java:488)

>> at org.apache.derby.impl.store.raw.data.StoredPage.releaseExclusive(StoredPage.java:1066)

>> at org.apache.derby.impl.store.raw.data.BasePage.unlatch(BasePage.java:1370) 
>> at
>> org.apache.derby.impl.store.raw.data.PageBasicOperation.releaseResource(PageBasicOperation.java:195)
>> at
>> org.apache.derby.impl.store.raw.data.PhysicalUndoOperation.releaseResource(PhysicalUndoOperation.java:177)
>> at org.apache.derby.impl.store.raw.log.FileLogger.undo(FileLogger.java:1055) 
>> at org.apache.derby.impl.store.raw.xact.Xact.abort(Xact.java:952) 
>> at
>> org.apache.derby.impl.store.raw.xact.XactFactory.rollbackAllTransactions(XactFactory.java:547)
>> at org.apache.derby.impl.store.raw.log.LogToFile.recover(LogToFile.java:1229) 
>> at org.apache.derby.impl.store.raw.RawStore.boot(RawStore.java:339) 
>> at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1982)

>> at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:334)

>> at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:531)

>> at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427)

>> at org.apache.derby.impl.store.access.RAMAccessManager.boot(RAMAccessManager.java:1019)

>> at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1982)

>> at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:334)

>> at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:531)

>> at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427)

>> at org.apache.derby.impl.db.BasicDatabase.bootStore(BasicDatabase.java:749) 
>> at org.apache.derby.impl.db.BasicDatabase.boot(BasicDatabase.java:177) 
>> at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1982)

>> at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:334)

>> at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(BaseMonitor.java:1816)

>> at
>> org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(BaseMonitor.java:1682)
>> at
>> org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(BaseMonitor.java:1560)
>> at
>> org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(BaseMonitor.java:979)
>> at
>> org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Monitor.java:550)
>> at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(EmbedConnection.java:2697)

>> ... 9 more 
>> 
>> 
>> Thanks,
>> Tim Wu


Mime
View raw message