db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bergquist, Brett" <BBergqu...@canoga.com>
Subject Derby received an error "ERROR XSDG0: Page Page(1325564,Container(0, 30832)) could not be read from disk."
Date Thu, 03 Sep 2015 22:35:22 GMT
A production system with a database of about 400G received this error today and it appears
that Derby shutdown parts of itself because from that point on it was spitting out errors
saying it could not find the database.   The system was restarted and came up clean and is
working with no issues inserting data at well over 100 inserts/second continuous and performing
queries and other data updates with no issues.

ERROR XSDG0: Page Page(1325564,Container(0, 30832)) could not be read from disk.
        at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
        at org.apache.derby.impl.store.raw.data.CachedPage.readPage(Unknown Source)
        at org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(Unknown Source)
        at org.apache.derby.impl.services.cache.ConcurrentCache.find(Unknown Source)
        at org.apache.derby.impl.store.raw.data.FileContainer.initPage(Unknown Source)
        at org.apache.derby.impl.store.raw.data.FileContainer.newPage(Unknown Source)
        at org.apache.derby.impl.store.raw.data.BaseContainer.addPage(Unknown Source)
        at org.apache.derby.impl.store.raw.data.BaseContainerHandle.addPage(Unknown Source)
        at org.apache.derby.impl.store.access.heap.HeapController.doInsert(Unknown Source)
        at org.apache.derby.impl.store.access.heap.HeapController.insertAndFetchLocation(Unknown
Source)
        at org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknown Source)
        at org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore(Unknown Source)
        at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown Source)
        at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
        at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)
        at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
        at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown Source)
        at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(Unknown Source)
        at org.apache.derby.iapi.jdbc.BrokeredPreparedStatement.execute(Unknown Source)
        at org.apache.derby.impl.drda.DRDAStatement.execute(Unknown Source)
        at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTTobjects(Unknown Source)
        at org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTT(Unknown Source)
        at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
        at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
Caused by: java.io.EOFException: Reached end of file while attempting to read a whole page.
        at org.apache.derby.impl.store.raw.data.RAFContainer4.readFull(Unknown Source)
        at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage0(Unknown Source)
        at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage(Unknown Source)
        at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage(Unknown S:

derby the reported errors like:

Thu Sep 03 16:03:54 GMT 2015 Thread[DRDAConnThread_80,5,main] (DATABASE = csemdb), (DRDAID
= ????????.??-1011901099978582906{5640754}),
Thu Sep 03 16:03:54 GMT 2015 :
org.apache.derby.iapi.error.ShutdownException:
        at org.apache.derby.iapi.services.context.ContextManager.checkInterrupt(Unknown Source)
        at org.apache.derby.iapi.services.context.ContextManager.popContext(Unknown Source)
        at org.apache.derby.iapi.services.context.ContextImpl.popMe(Unknown Source)
        at org.apache.derby.jdbc.EmbedXAResource.removeXATransaction(Unknown Source)
        at org.apache.derby.jdbc.EmbedXAResource.returnConnectionToResource(Unknown Source)
        at org.apache.derby.jdbc.EmbedXAResource.commit(Unknown Source)
        at org.apache.derby.impl.drda.DRDAXAProtocol.commitXATransaction(Unknown Source)
        at org.apache.derby.impl.drda.DRDAXAProtocol.commitTransaction(Unknown Source)
        at org.apache.derby.impl.drda.DRDAXAProtocol.parseSYNCCTL(Unknown Source)
        at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
        at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
Thu Sep 03 16:03:54 GMT 2015 Thread[DRDAConnThread_114,5,main] (DATABASE = csemdb), (DRDAID
= ????????.??-940969405847491181{5640766}), null
Thu Sep 03 16:03:54 GMT 2015 : null
Thu Sep 03 16:03:54 GMT 2015 : null
java.lang.NullPointerException
        at org.apache.derby.impl.drda.DRDAConnThread.writePBSD(Unknown Source)
        at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
        at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
java.lang.NullPointerException
        at org.apache.derby.impl.drda.DRDAConnThread.writePBSD(Unknown Source)
        at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
        at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
Thu Sep 03 16:03:54 GMT 2015 Thread[DRDAConnThread_191,5,main] (DATABASE = csemdb), (DRDAID
= ????????.??-939843505940648198{5640770}), null
Thu Sep 03 16:03:54 GMT 2015 : null

And

Thu Sep 03 16:03:54 GMT 2015 Thread[DRDAConnThread_185,5,main] Cleanup action starting
java.sql.SQLException: Database 'csemdb' not found.
        at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
        at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
        at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
        at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
        at org.apache.derby.impl.jdbc.EmbedConnection.newSQLException(Unknown Source)
        at org.apache.derby.impl.jdbc.EmbedConnection.handleDBNotFound(Unknown Source)
        at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
        at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(Unknown Source)
        at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source)
        at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
        at org.apache.derby.jdbc.EmbeddedBaseDataSource.getConnection(Unknown Source)
        at org.apache.derby.jdbc.EmbedPooledConnection.openRealConnection(Unknown Source)
        at org.apache.derby.jdbc.EmbedXAConnection.getRealConnection(Unknown Source)
        at org.apache.derby.iapi.jdbc.BrokeredConnection.getRealConnection(Unknown Source)
        at org.apache.derby.iapi.jdbc.BrokeredConnection.isClosed(Unknown Source

The system is an Oracle M5000 Enterprise server with what I believe is a 15TB Sun ZFS Storage
7320 external ZFS storage array connected by Fibre Channel.   This is the first time in over
8 years we have seen any I/O error like such.

What I am trying to confirm is that this is really low level derby code that if it reports
an "java.io.EOFException" like it did, it really did have an I/O error somewhere in reading
the page from the container file.   Things like performance, java heap space, etc, can pretty
much be ruled out as causing such an error.   My gut feeling is that maybe something in the
connection to this storage array had a hiccup.   This setup is at the customer site and I
cannot directly access system logs nor do I have knowledge on how this storage array works
and how to look at such but just having confirmation that an I/O error really did occur would
help.



________________________________
Canoga Perkins
20600 Prairie Street
Chatsworth, CA 91311
(818) 718-6300

This e-mail and any attached document(s) is confidential and is intended only for the review
of the party to whom it is addressed. If you have received this transmission in error, please
notify the sender immediately and discard the original message and any attachment(s).

Mime
View raw message