db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brett Bergquist (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DERBY-5632) Logical deadlock happened when freezing/unfreezing the database
Date Mon, 10 Dec 2012 22:41:22 GMT

    [ https://issues.apache.org/jira/browse/DERBY-5632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528381#comment-13528381

Brett Bergquist commented on DERBY-5632:

This is still a major problem even when using the same connection, the database can get into
a state that the freeze works, but the unfreeze locks up.   

At a customer size, this procedure is done each night to backup the database.  Since the 9/1/2012,
this has failed 5 times, causing the database to remain in a frozen state with no possibility
of unfreezing it.   When in this state, no connections to the database can be created including
a connection to shut down the database.   Forcefully killing the network server is the only
way to recover.

Note that there is a background job in the system that is running an UPDATE_STATISTICS every
minute as in Derby, the automatic statistics update daemon has problems and cannot
be used.

> Logical deadlock happened when freezing/unfreezing the database
> ---------------------------------------------------------------
>                 Key: DERBY-5632
>                 URL: https://issues.apache.org/jira/browse/DERBY-5632
>             Project: Derby
>          Issue Type: Bug
>          Components: Documentation, Services
>    Affects Versions:
>         Environment: Oracle M3000/Solaris 10
>            Reporter: Brett Bergquist
>              Labels: derby_triage10_10
>         Attachments: stack.txt
> Tried to make a quick database backup by freezing the database, performing a ZFS snapshot,
and then unfreezing the database.   The database was frozen but then a connection to the database
could not be established to unfreeze the database.
> Looking at the stack trace of the network server, , I see 3 threads that are trying to
process a connection request.   Each of these is waiting on:
>                 at org.apache.derby.impl.store.access.RAMAccessManager.conglomCacheFind(Unknown
>                 - waiting to lock <0xfffffffd3a7fcc68> (a org.apache.derby.impl.services.cache.ConcurrentCache)
> That object is owned by:
>                 - locked <0xfffffffd3a7fcc68> (a org.apache.derby.impl.services.cache.ConcurrentCache)
>                 at org.apache.derby.impl.store.access.RAMTransaction.findExistingConglomerate(Unknown
>                 at org.apache.derby.impl.store.access.RAMTransaction.openGroupFetchScan(Unknown
>                 at org.apache.derby.impl.services.daemon.IndexStatisticsDaemonImpl.updateIndexStatsMinion(Unknown
>                 at org.apache.derby.impl.services.daemon.IndexStatisticsDaemonImpl.runExplicitly(Unknown
>                 at org.apache.derby.impl.sql.execute.AlterTableConstantAction.updateStatistics(Unknown
> which itself is waiting for the object:
>                 at java.lang.Object.wait(Native Method)
>                 - waiting on <0xfffffffd3ac1d608> (a org.apache.derby.impl.store.raw.log.LogToFile)
>                 at java.lang.Object.wait(Object.java:485)
>                 at org.apache.derby.impl.store.raw.log.LogToFile.flush(Unknown Source)
>                 - locked <0xfffffffd3ac1d608> (a org.apache.derby.impl.store.raw.log.LogToFile)
>                 at org.apache.derby.impl.store.raw.log.LogToFile.flush(Unknown Source)
>                 at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.flush(Unknown
> So basically what I think is happening is that the database is frozen, the statistics
are being updated on another thread which has the "org.apache.derby.impl.services.cache.ConcurrentCache"
locked and then waits for the LogToFile lock and the connecting threads are waiting to lock
"org.apache.derby.impl.services.cache.ConcurrentCache" to connect and these are where the
database is going to be unfrozen.    Not a deadlock as far as the JVM is concerned but it
will never leave this state either.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message