Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: <derby-dev@db.apache.org>
Date: Fri, 1 Jun 2012 10:41:23 +0000 (UTC)
From: "Knut Anders Hatlen (JIRA)" <jira@apache.org>
To: derby-dev@db.apache.org
Message-ID: <289149131.26179.1338547283226.JavaMail.jiratomcat@issues-vm>
In-Reply-To: 
 <1460484318.18151.1311930909838.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (DERBY-5358) SYSCS_COMPRESS_TABLE failed with
 conglomerate not found exception
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/DERBY-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13287320#comment-13287320 ] 

Knut Anders Hatlen commented on DERBY-5358:
-------------------------------------------

It looks like only the fix in TableDescriptor is needed for this
particular problem, by the way. The repro is running fine without the
change in ConglomerateDescriptor.

I'm not sure whether this is a general synchronization issue in
TableDescriptor, or just a problem with the one specific method
getHeapConglomerateId(). The other occurrences of fields being
modified, seem to be in setter methods or other methods whose names
suggest they will be modifying the descriptor. I would assume that the
callers of those methods have obtained an exclusive table lock, or use
some other mechanism to ensure exclusive access, before they call the
methods.

What's so special about getHeapConglomerateId(), is that the write
happens inside a method whose name suggests it'll just do a read,
because of lazy initialization of the field. The callers cannot
reasonably be expected to know this and obtain an exclusive table lock
just to fetch the id of the heap conglomerate, so extra protection is
needed for this method.

As to when this could happen, I think it could happen any time two
threads concurrently ask for the conglomerate id when no other thread
has done the same before. It could be as simple as two threads
selecting from the same table right after the database has been booted
(although that particular may be hidden by the last fix that went into
DERBY-5406, where we'd re-try if a compilation fails with conglomerate
not found).

Also, after any DDL operation, the TD cache in the data dictionary is
cleared, so any operation that needs a TD after a DDL operation, also
if the DDL operation didn't touch the table described by the TD, will
get an instance whose heapConglomerateNumber field is uninitialized.

So, theoretically, any two threads accessing the same table after a
DDL operation, could encounter this problem.
                
> SYSCS_COMPRESS_TABLE failed with conglomerate not found exception
> -----------------------------------------------------------------
>
>                 Key: DERBY-5358
>                 URL: https://issues.apache.org/jira/browse/DERBY-5358
>             Project: Derby
>          Issue Type: Bug
>          Components: Store
>    Affects Versions: 10.9.1.0
>            Reporter: Knut Anders Hatlen
>            Assignee: Knut Anders Hatlen
>              Labels: derby_triage10_9
>         Attachments: volatile.diff
>
>
> When running the D4275.java repro attached to DERBY-4275 (with the patch invalidate-during-invalidation.diff as well as the fix for DERBY-5161 to prevent the select thread from failing) in four parallel processes on the same machine, one of the processes failed with the following stack trace:
> java.sql.SQLException: The exception 'java.sql.SQLException: The conglomerate (4,294,967,295) requested does not exist.' was thrown while evaluating an expression.
>         at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:98)
>         at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Util.java:142)
>         at org.apache.derby.impl.jdbc.Util.seeNextException(Util.java:278)
>         at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(TransactionResourceImpl.java:407)
>         at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(TransactionResourceImpl.java:348)
>         at org.apache.derby.impl.jdbc.EmbedConnection.handleException(EmbedConnection.java:2290)
>         at org.apache.derby.impl.jdbc.ConnectionChild.handleException(ConnectionChild.java:82)
>         at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedStatement.java:1334)
>         at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(EmbedPreparedStatement.java:1686)
>         at org.apache.derby.impl.jdbc.EmbedPreparedStatement.execute(EmbedPreparedStatement.java:1341)
>         at D4275.main(D4275.java:52)
> Caused by: java.sql.SQLException: The exception 'java.sql.SQLException: The conglomerate (4,294,967,295) requested does not exist.' was thrown while evaluating an expression.
>         at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(SQLExceptionFactory.java:45)
>         at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(SQLExceptionFactory40.java:122)
>         at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:71)
>         ... 10 more
> Caused by: java.sql.SQLException: The conglomerate (4,294,967,295) requested does not exist.
>         at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(SQLExceptionFactory.java:45)
>         at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(SQLExceptionFactory40.java:122)
>         at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:71)
>         at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Util.java:256)
>         at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(TransactionResourceImpl.java:400)
>         at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(TransactionResourceImpl.java:348)
>         at org.apache.derby.impl.jdbc.EmbedConnection.handleException(EmbedConnection.java:2290)
>         at org.apache.derby.impl.jdbc.ConnectionChild.handleException(ConnectionChild.java:82)
>         at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedStatement.java:1334)
>         at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(EmbedPreparedStatement.java:1686)
>         at org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(EmbedPreparedStatement.java:308)
>         at org.apache.derby.catalog.SystemProcedures.SYSCS_COMPRESS_TABLE(SystemProcedures.java:792)
>         at org.apache.derby.exe.acd381409ax0131x72b6x8e11x0000037164a81.g0(Unknown Source)
>         at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(ReflectMethod.java:46)
>         at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(CallStatementResultSet.java:75)
>         at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(GenericPreparedStatement.java:448)
>         at org.apache.derby.impl.sql.GenericPreparedStatement.execute(GenericPreparedStatement.java:319)
>         at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedStatement.java:1242)
>         ... 3 more
> Caused by: ERROR XSAI2: The conglomerate (4,294,967,295) requested does not exist.
>         at org.apache.derby.iapi.error.StandardException.newException(StandardException.java:278)
>         at org.apache.derby.impl.store.access.RAMAccessManager.getFactoryFromConglomId(RAMAccessManager.java:382)
>         at org.apache.derby.impl.store.access.RAMAccessManager.conglomCacheFind(RAMAccessManager.java:482)
>         at org.apache.derby.impl.store.access.RAMTransaction.findExistingConglomerate(RAMTransaction.java:394)
>         at org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(RAMTransaction.java:1308)
>         at org.apache.derby.impl.sql.execute.DDLConstantAction.lockTableForDDL(DDLConstantAction.java:252)
>         at org.apache.derby.impl.sql.execute.AlterTableConstantAction.executeConstantActionBody(AlterTableConstantAction.java:364)
>         at org.apache.derby.impl.sql.execute.AlterTableConstantAction.executeConstantAction(AlterTableConstantAction.java:275)
>         at org.apache.derby.impl.sql.execute.MiscResultSet.open(MiscResultSet.java:61)
>         at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(GenericPreparedStatement.java:448)
>         at org.apache.derby.impl.sql.GenericPreparedStatement.execute(GenericPreparedStatement.java:319)
>         at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(EmbedStatement.java:1242)
>         ... 15 more
> Test stopped after 9342310 ms
> The conglomerate number 4,294,967,295 looks suspicious, as it's equal to 2^32-1. Perhaps it's hitting some internal limit on the number of conglomerates? The test case used the in-memory back-end.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira