Return-Path: X-Original-To: apmail-db-derby-commits-archive@www.apache.org Delivered-To: apmail-db-derby-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 78A539854 for ; Tue, 17 Apr 2012 18:40:42 +0000 (UTC) Received: (qmail 29720 invoked by uid 500); 17 Apr 2012 18:40:42 -0000 Delivered-To: apmail-db-derby-commits-archive@db.apache.org Received: (qmail 29701 invoked by uid 500); 17 Apr 2012 18:40:42 -0000 Mailing-List: contact derby-commits-help@db.apache.org; run by ezmlm Precedence: bulk list-help: list-unsubscribe: List-Post: Reply-To: "Derby Development" List-Id: Delivered-To: mailing list derby-commits@db.apache.org Received: (qmail 29680 invoked by uid 99); 17 Apr 2012 18:40:42 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 Apr 2012 18:40:42 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 Apr 2012 18:40:38 +0000 Received: from eris.apache.org (localhost [127.0.0.1]) by eris.apache.org (Postfix) with ESMTP id 6B5FD238897D; Tue, 17 Apr 2012 18:40:16 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1327218 - in /db/derby/code/trunk/java: engine/org/apache/derby/iapi/store/access/ engine/org/apache/derby/impl/sql/catalog/ engine/org/apache/derby/impl/sql/conn/ engine/org/apache/derby/impl/sql/execute/ engine/org/apache/derby/impl/stor... Date: Tue, 17 Apr 2012 18:40:16 -0000 To: derby-commits@db.apache.org From: mikem@apache.org X-Mailer: svnmailer-1.0.8-patched Message-Id: <20120417184016.6B5FD238897D@eris.apache.org> Author: mikem Date: Tue Apr 17 18:40:15 2012 New Revision: 1327218 URL: http://svn.apache.org/viewvc?rev=1327218&view=rev Log: DERBY-5494 Prior to this fix the nested user update transaction used by sequence updater was doing a "lazy" commit, where the log record for the commit waw written to the stream but not forced to disk. It would get forced to disk by any subsequent user transaction commit. Changed system to default doing a real commit for all nested user update transactions, and those that don't need that behavior should use commitNoSync(). Changed identity columns to use the commitNoSync() to keep same performance for those operations. Includes the test contributed by rickh as part of DERBY-5493 change. Modified: db/derby/code/trunk/java/engine/org/apache/derby/iapi/store/access/TransactionController.java db/derby/code/trunk/java/engine/org/apache/derby/impl/sql/catalog/SequenceUpdater.java db/derby/code/trunk/java/engine/org/apache/derby/impl/sql/conn/GenericLanguageConnectionContext.java db/derby/code/trunk/java/engine/org/apache/derby/impl/sql/execute/InsertResultSet.java db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/xact/XactFactory.java db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/SequenceGeneratorTest.java Modified: db/derby/code/trunk/java/engine/org/apache/derby/iapi/store/access/TransactionController.java URL: http://svn.apache.org/viewvc/db/derby/code/trunk/java/engine/org/apache/derby/iapi/store/access/TransactionController.java?rev=1327218&r1=1327217&r2=1327218&view=diff ============================================================================== --- db/derby/code/trunk/java/engine/org/apache/derby/iapi/store/access/TransactionController.java (original) +++ db/derby/code/trunk/java/engine/org/apache/derby/iapi/store/access/TransactionController.java Tue Apr 17 18:40:15 2012 @@ -719,6 +719,33 @@ public interface TransactionController * Once the first write of a non-readOnly nested transaction is done, * then the nested user transaction must be committed or aborted before * any write operation is attempted in the parent transaction. + * (p> + * fix for DERBY-5493 introduced a behavior change for commits executed + * against an updatable nested user transaction. Prior to this change + * commits would execute a "lazy" commit where commit log record would only + * be written to the stream, not guaranteed to disk. After this change + * commits on these transactions will always be forced to disk. To get + * the previous behavior one must call commitNoSync() instead. + *

+ * examples of current usage of nested updatable transactions in Derby + * include: + * o recompile and saving of stored prepared statements, changed with + * DERBY-5493 to do synchronous commit. Code in SPSDescriptor.java. + * o sequence updater reserves new "range" of values in sequence + * catalog, changed with DERBY-5493 to do synchronous commit. Without + * this change crash of system might lose the updat of the range and + * then return same value on reboot. Code in SequenceUpdater.java + * o in place compress defragment phase committing units of work in + * moving tuples around in heap and indexes. changed with DERBY-5493 + * to do synchronous commit. code in AlterTableConstantAction.java. + * o used for creation of users initial default schema in SYSSCHEMAS. + * moving tuples around in heap and indexes. changed with DERBY-5493 + * to do synchronous commit. code in DDLConstantAction.java. + * o autoincrement/generated key case. Kept behavior previous to + * DERBY-5493 by changing to use commitNoSync. Changing every + * key allocation to be a synchronous commit would be a huge performance + * problem for existing applications depending on current performance. + * code in InsertResultSet.java * * @param readOnly Is transaction readonly? Only 1 non-readonly nested * transaction is allowed per transaction. Modified: db/derby/code/trunk/java/engine/org/apache/derby/impl/sql/catalog/SequenceUpdater.java URL: http://svn.apache.org/viewvc/db/derby/code/trunk/java/engine/org/apache/derby/impl/sql/catalog/SequenceUpdater.java?rev=1327218&r1=1327217&r2=1327218&view=diff ============================================================================== --- db/derby/code/trunk/java/engine/org/apache/derby/impl/sql/catalog/SequenceUpdater.java (original) +++ db/derby/code/trunk/java/engine/org/apache/derby/impl/sql/catalog/SequenceUpdater.java Tue Apr 17 18:40:15 2012 @@ -477,6 +477,10 @@ public abstract class SequenceUpdater im } finally { + // DERBY-5494, if this commit does not flush log then an + // unorderly shutdown could lose the update. Do not use + // commitNoSync(), and store needs to flush user nested update + // transaction commits by default. nestedTransaction.commit(); nestedTransaction.destroy(); } Modified: db/derby/code/trunk/java/engine/org/apache/derby/impl/sql/conn/GenericLanguageConnectionContext.java URL: http://svn.apache.org/viewvc/db/derby/code/trunk/java/engine/org/apache/derby/impl/sql/conn/GenericLanguageConnectionContext.java?rev=1327218&r1=1327217&r2=1327218&view=diff ============================================================================== --- db/derby/code/trunk/java/engine/org/apache/derby/impl/sql/conn/GenericLanguageConnectionContext.java (original) +++ db/derby/code/trunk/java/engine/org/apache/derby/impl/sql/conn/GenericLanguageConnectionContext.java Tue Apr 17 18:40:15 2012 @@ -1886,8 +1886,19 @@ public class GenericLanguageConnectionCo // method support read-write nested transactions as well // instead of callers using the startNestedUserTransaction // directly on tran. + if (SanityManager.DEBUG) + { + // if called for update transactions, compile would start using + // non-readonly xacts for compile. For now, throw an error if + // someone tries to use this call to make non readonly transaction. + SanityManager.ASSERT( + readOnly, + "Routine not yet coded to support non-readonly transactions."); + } + if (readOnlyNestedTransaction == null) readOnlyNestedTransaction = tran.startNestedUserTransaction(readOnly); + queryNestingDepth++; } Modified: db/derby/code/trunk/java/engine/org/apache/derby/impl/sql/execute/InsertResultSet.java URL: http://svn.apache.org/viewvc/db/derby/code/trunk/java/engine/org/apache/derby/impl/sql/execute/InsertResultSet.java?rev=1327218&r1=1327217&r2=1327218&view=diff ============================================================================== --- db/derby/code/trunk/java/engine/org/apache/derby/impl/sql/execute/InsertResultSet.java (original) +++ db/derby/code/trunk/java/engine/org/apache/derby/impl/sql/execute/InsertResultSet.java Tue Apr 17 18:40:15 2012 @@ -786,7 +786,6 @@ class InsertResultSet extends DMLWriteRe cd.getColumnName()); aiCache[columnPosition - 1].setValue(ret); } - else { NumberDataValue newValue; @@ -797,7 +796,6 @@ class InsertResultSet extends DMLWriteRe nestedTC = tc.startNestedUserTransaction(false); tcToUse = nestedTC; } - catch (StandardException se) { // If I cannot start a Nested User Transaction use the parent @@ -815,7 +813,6 @@ class InsertResultSet extends DMLWriteRe constants.autoincRowLocation[index], tcToUse, true, aiCache[index], (tcToUse == tc)); } - catch (StandardException se) { if (tcToUse == tc) @@ -851,9 +848,21 @@ class InsertResultSet extends DMLWriteRe // no matter what, commit the nested transaction; if something // bad happened in the child xaction lets not abort the parent // here. + if (nestedTC != null) { - nestedTC.commit(); + // DERBY-5493 - prior to fix all nested user update + // transactions did a nosync commit when commit() was + // called, this default has been changed to do synced + // commit. Changed this commit to be commitNoSync to + // not introduce performce degredation for autoincrement + // keys. As before, if server crashes the changes + // made in the nested transaction may be lost. If any + // subsequent user transaction is commited, including any + // inserts that would depend on the autoincrement value + // change then the nested tranaction is guaranteed on + // system crash. + nestedTC.commitNoSync(TransactionController.RELEASE_LOCKS); nestedTC.destroy(); } } Modified: db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/xact/XactFactory.java URL: http://svn.apache.org/viewvc/db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/xact/XactFactory.java?rev=1327218&r1=1327217&r2=1327218&view=diff ============================================================================== --- db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/xact/XactFactory.java (original) +++ db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/xact/XactFactory.java Tue Apr 17 18:40:15 2012 @@ -943,12 +943,18 @@ public class XactFactory implements Tran public boolean flushLogOnCommit(String contextName) { // - // if this is a user transaction, flush the log + // if this is a user transaction, flush the log by default. + // if this is a nested user update transaction, flush log by default. // if this is an internal or nested top transaction, do not // flush, let it age out. + // + // In all cases log will not be flushsed by Xact.prepareCommit() + // if commitNoSync() has been called rather than commit. // - return (contextName == USER_CONTEXT_ID || - contextName.equals(USER_CONTEXT_ID)); + return (contextName == USER_CONTEXT_ID || + contextName.equals(USER_CONTEXT_ID) || + contextName == NESTED_UPDATE_USER_CONTEXT_ID || + contextName.equals(NESTED_UPDATE_USER_CONTEXT_ID)); } Modified: db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/SequenceGeneratorTest.java URL: http://svn.apache.org/viewvc/db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/SequenceGeneratorTest.java?rev=1327218&r1=1327217&r2=1327218&view=diff ============================================================================== --- db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/SequenceGeneratorTest.java (original) +++ db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/lang/SequenceGeneratorTest.java Tue Apr 17 18:40:15 2012 @@ -680,6 +680,56 @@ public class SequenceGeneratorTest exte seq_10_upperBound = seq_10_value + ALLOCATION_COUNT; vetBumping( conn, TEST_DBO, "SEQ_10", seq_10_value++, seq_10_upperBound ); } + + /** + *

+ * Verify that system crash does not rollback changes to SYSSEQUENCES.CURRENTVALUE. + * See DERBY-5494. + *

+ */ + public void test_13_5494() throws Exception + { + String dbName = "DB_5494"; + + // create a sequence and get the first value from it, then crash + assertLaunchedJUnitTestMethod( getClass().getName() + ".preCrashActions", dbName ); + + // now check that the sequence state was correctly recovered + assertLaunchedJUnitTestMethod( getClass().getName() + ".postCrashActions", dbName ); + } + // actions to perform just before a crash + public void preCrashActions() throws Exception + { + Connection dboConn = openUserConnection( TEST_DBO ); + Connection ruthConn = openUserConnection( "RUTH" ); + int initialValue = Integer.MIN_VALUE; + + goodStatement( dboConn, "create sequence s_5494\n" ); + + assertNextValue( dboConn, TEST_DBO, "S_5494", initialValue ); + + assertEquals( (long) (initialValue + ALLOCATION_COUNT), getCurrentValue( ruthConn, TEST_DBO, "S_5494" ) ); + } + // actions to perform after the crash + public void postCrashActions() throws Exception + { + int initialValue = (int) (Integer.MIN_VALUE + ALLOCATION_COUNT); + + // now verify that, after the crash, SYSSEQUENCES has still been advanced + Connection dboConn = openUserConnection( TEST_DBO ); + assertEquals( (long) initialValue, getCurrentValue( dboConn, TEST_DBO, "S_5494" ) ); + + assertNextValue( dboConn, TEST_DBO, "S_5494", initialValue ); + + goodStatement( dboConn, "drop sequence s_5494 restrict\n" ); + } + private void assertNextValue( Connection conn, String schema, String sequenceName, int expectedValue ) + throws Exception + { + PreparedStatement ps = chattyPrepare( conn, "values( next value for " + schema + "." + sequenceName + " )\n" ); + + assertEquals( expectedValue, getScalarInteger( ps ) ); + } /////////////////////////////////////////////////////////////////////////////////// // @@ -705,6 +755,26 @@ public class SequenceGeneratorTest exte return retval; } + + /** Get the current value from a sequence */ + private long getCurrentValue( + Connection conn, + String schemaName, + String sequenceName ) + throws Exception + { + PreparedStatement ps = chattyPrepare + ( conn, + "select currentvalue from sys.syssequences seq, sys.sysschemas s where s.schemaname = ? and seq.sequencename = ? and s.schemaid = seq.schemaid" ); + ps.setString( 1, schemaName ); + ps.setString( 2, sequenceName ); + + long retval = getScalarLong( ps ); + + conn.commit(); + + return retval; + } /** Get a scalar integer result from a query */ private int getScalarInteger( PreparedStatement ps ) throws Exception