db-derby-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mi...@apache.org
Subject svn commit: r1292079 - in /db/derby/code/trunk/java: engine/org/apache/derby/impl/store/raw/data/DropOnCommit.java engine/org/apache/derby/impl/store/raw/xact/Xact.java testing/org/apache/derbyTesting/functionTests/tests/largedata/_Suite.java
Date Tue, 21 Feb 2012 23:50:18 GMT
Author: mikem
Date: Tue Feb 21 23:50:17 2012
New Revision: 1292079

URL: http://svn.apache.org/viewvc?rev=1292079&view=rev
Log:
DERBY-5624 System can run out of stack space while processing DropOnCommit requests.

Taking care of cleanup after a commit is handled by notifying all "Observers"
that an event has taken place that they might want to act on and cleanup.  In
the added test case this is triggered by off line commit which effectively
drops and recreates the base table and all of its indexes after loading the
data into them.  

Sometimes these Observers may execute work which adds to the Observer queue,
and that queue can "miss" them in the first pass through.

A previous fix for this problem added a recursive call to notifyObservers in
the place that could cause this addition of observers.  This recursive call
was causing stack problems when the number of Observers became large.  For
the checked in test case this was 1000 indexes on 1000 columns of the table.
For other users I believe the cause was a by product of sorts on large disk
based sorts for multi-gigabyte tables and indexes.  2 users were reporting
similar failed stacks for failing compresses of large tables, and one was
able to take this fix to their environment and then successfully run the
compress.

The fix was to remove the recursion and instead loop at the outermost point
until there were no Observers.

Adding the test to the largedata suite as it takes over 10 minutes to run
on my machine.


Modified:
    db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/data/DropOnCommit.java
    db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/xact/Xact.java
    db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/largedata/_Suite.java

Modified: db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/data/DropOnCommit.java
URL: http://svn.apache.org/viewvc/db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/data/DropOnCommit.java?rev=1292079&r1=1292078&r2=1292079&view=diff
==============================================================================
--- db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/data/DropOnCommit.java
(original)
+++ db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/data/DropOnCommit.java
Tue Feb 21 23:50:17 2012
@@ -98,14 +98,6 @@ public class DropOnCommit extends Contai
 			}
 
 			obj.deleteObserver(this);
-
-            // DERBY-3993
-            // make sure any observer that may have been added by either
-            // dropContainer() or dropStreamContainer() is also handled.
-            // The calling notifyObservers() call from Xact.doComplete()
-            // may not "see" new observers added during processing of the
-            // initial observer list.
-            xact.notifyObservers(arg);
 		}
 	}
 }

Modified: db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/xact/Xact.java
URL: http://svn.apache.org/viewvc/db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/xact/Xact.java?rev=1292079&r1=1292078&r2=1292079&view=diff
==============================================================================
--- db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/xact/Xact.java (original)
+++ db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/xact/Xact.java Tue Feb
21 23:50:17 2012
@@ -1960,20 +1960,25 @@ public class Xact extends RawTransaction
 		if (savePoints != null)
 			savePoints.removeAllElements();
 
-		// notify any of our observers that we are completing.
-		notifyObservers(commitOrAbort);
+        do
+        {
+            // notify any of our observers that we are completing.
+            notifyObservers(commitOrAbort);
 
-		checkObserverException();
+            checkObserverException();
 
-		if (SanityManager.DEBUG) 
-        {
-			if (countObservers() != 0)
-            {
-                SanityManager.THROWASSERT(
-                    "There should be 0 observers, but we still have "
-					+ countObservers() + " observers.");
-            }
-		}
+            // DERBY-3993
+            // make sure any observer that may have been added by either
+            // dropContainer() or dropStreamContainer() is also handled.
+            // The calling notifyObservers() call from Xact.doComplete()
+            // may not "see" new observers added during processing of the
+            // initial observer list.  So loop until notifyObservers()
+            // call causes countObservers() to go to 0.  This should only
+            // loop if one of the observers adds to the list as part of
+            // the notify.  Even then depending on ordering the added
+            // observer may be picked up in the first try.
+
+        } while (countObservers() > 0);
 	}
 
 	private void checkObserverException() throws StandardException {

Modified: db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/largedata/_Suite.java
URL: http://svn.apache.org/viewvc/db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/largedata/_Suite.java?rev=1292079&r1=1292078&r2=1292079&view=diff
==============================================================================
--- db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/largedata/_Suite.java
(original)
+++ db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/largedata/_Suite.java
Tue Feb 21 23:50:17 2012
@@ -40,6 +40,7 @@ public class _Suite extends BaseJDBCTest
     public static Test suite() {
         TestSuite suite = new TestSuite("largedata suite");
         suite.addTest(LobLimitsLiteTest.suite());
+        suite.addTest(Derby5624Test.suite());
         suite.addTest(LobLimitsTest.suite());
         suite.addTest(LobLimitsClientTest.suite());
         return suite;



Mime
View raw message