db-derby-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mi...@apache.org
Subject svn commit: r357269 - in /db/derby/code/trunk/java: engine/org/apache/derby/iapi/store/raw/xact/ engine/org/apache/derby/impl/store/raw/data/ engine/org/apache/derby/impl/store/raw/xact/ testing/org/apache/derbyTesting/functionTests/master/ testing/org...
Date Sat, 17 Dec 2005 01:52:17 GMT
Author: mikem
Date: Fri Dec 16 17:52:03 2005
New Revision: 357269

URL: http://svn.apache.org/viewcvs?rev=357269&view=rev
Log:
DERBY-239, committing on behalf of Suresh Thalamati.

This patch adds code to support online backup when  jar operations 
are running parallel to the backup. Jar files are not logged, but the 
system catalogs updates are logged when a jar file is added/replaced. 
If the jar file operations are allowed during the backup, system catalog 
(sys.sysfiles) table in the backup database can have a reference to a jar file 
that does not exist in the backup database. And also backup can contain 
partial written jar files. To make a consistent online backup, this patch:

1) Makes Backup operation wait/fail for all the jar operations activity in progress to complete.
2) Blocks jar file operations when a backup is in progress.

This patch also adds a new test to test the online backup 
with jar operations.



Added:
    db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/master/OnlineBackupTest3.out
    db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/OnlineBackupTest3.java
    db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/OnlineBackupTest3_app.properties
    db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/obtest_customer.jar
  (with props)
Modified:
    db/derby/code/trunk/java/engine/org/apache/derby/iapi/store/raw/xact/RawTransaction.java
    db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/data/BaseDataFileFactory.java
    db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/data/RFResource.java
    db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/xact/Xact.java
    db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/xact/XactFactory.java
    db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/suites/storemore.runall
    db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/copyfiles.ant

Modified: db/derby/code/trunk/java/engine/org/apache/derby/iapi/store/raw/xact/RawTransaction.java
URL: http://svn.apache.org/viewcvs/db/derby/code/trunk/java/engine/org/apache/derby/iapi/store/raw/xact/RawTransaction.java?rev=357269&r1=357268&r2=357269&view=diff
==============================================================================
--- db/derby/code/trunk/java/engine/org/apache/derby/iapi/store/raw/xact/RawTransaction.java
(original)
+++ db/derby/code/trunk/java/engine/org/apache/derby/iapi/store/raw/xact/RawTransaction.java
Fri Dec 16 17:52:03 2005
@@ -329,14 +329,20 @@
 		throws StandardException;
 
 	
-	/*
-	 * Try setting  the transaction to be in backup blocking state.
-	 *
-	 * @return     <tt>true</tt> if the transaction can be set to a 
-	 *             blocking state. 
-	 *             <tt>false</tt> otherwise.
-	 */
-	public abstract boolean setBackupBlockingState();
+    /*
+     * Try setting the transaction to be in backup blocking state.
+     *
+     * @param wait if <tt>true</tt>, waits until the transaction
+     *             can be set into backup blocking state.
+     * @return     <tt>true</tt> if the transaction can be set to a 
+     *             blocking state. 
+     *             <tt>false</tt> otherwise.
+     * @exception StandardException if interrupted while waiting 
+     *            for backup to complete to set the transaction into
+     *            backup blocking state.
+     */
+    public abstract boolean setBackupBlockingState(boolean wait)
+        throws StandardException;
 
 }
 

Modified: db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/data/BaseDataFileFactory.java
URL: http://svn.apache.org/viewcvs/db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/data/BaseDataFileFactory.java?rev=357269&r1=357268&r2=357269&view=diff
==============================================================================
--- db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/data/BaseDataFileFactory.java
(original)
+++ db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/data/BaseDataFileFactory.java
Fri Dec 16 17:52:03 2005
@@ -654,15 +654,15 @@
 
 					// block the online backup if the container is being 
 					// opened in unlogged mode, if the backup is already 
-					// running then convert all unlogged opens to a logged ones,
-					// otherwise onlibe backup copy will be inconsistent.
+					// running then convert all unlogged opens to logged ones,
+					// otherwise online backup copy will be inconsistent.
 
 					if (((mode & ContainerHandle.MODE_UNLOGGED) == 
 						 ContainerHandle.MODE_UNLOGGED) || 
 						((mode & ContainerHandle.MODE_CREATE_UNLOGGED) == 
 						 ContainerHandle.MODE_CREATE_UNLOGGED))									   
 					{
-						if(!t.setBackupBlockingState()) {
+						if (!t.setBackupBlockingState(false)) {
 							// when a backup is in progress transaction can not
                             // be set to backup blocking state, so convert 
                             // unlogged opens to logged mode.

Modified: db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/data/RFResource.java
URL: http://svn.apache.org/viewcvs/db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/data/RFResource.java?rev=357269&r1=357268&r2=357269&view=diff
==============================================================================
--- db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/data/RFResource.java (original)
+++ db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/data/RFResource.java Fri
Dec 16 17:52:03 2005
@@ -31,6 +31,7 @@
 import org.apache.derby.iapi.store.raw.Transaction;
 import org.apache.derby.iapi.store.access.AccessFactoryGlobals;
 import org.apache.derby.iapi.store.access.DatabaseInstant;
+import org.apache.derby.iapi.store.raw.xact.RawTransaction;
 
 import org.apache.derby.io.StorageFactory;
 import org.apache.derby.io.WritableStorageFactory;
@@ -75,6 +76,19 @@
                         SQLState.FILE_EXISTS, file);
             }
 
+            ContextManager cm = 
+                ContextService.getFactory().getCurrentContextManager();
+
+            Transaction tran = 
+                factory.getRawStoreFactory().findUserTransaction(
+                        cm, AccessFactoryGlobals.USER_TRANS_NAME);
+            
+            // Prevent backup operation when a jar file is being added
+            // by setting the transaction into a backup blocking state.
+            // If backup is already in progress this call will wait 
+            // for the backup to finish .
+            ((RawTransaction)tran).setBackupBlockingState(true);
+
 			StorageFile directory = file.getParentDir();
             if (!directory.exists())
 			{
@@ -142,6 +156,12 @@
             factory.getRawStoreFactory().findUserTransaction(
                 cm, AccessFactoryGlobals.USER_TRANS_NAME);
 
+        // Prevent backup operation when a jar file is being removed
+        // by setting the transaction into a backup blocking state.
+        // If backup is already in progress this call will wait 
+        // for the backup to finish.
+        ((RawTransaction)tran).setBackupBlockingState(true);
+
 		tran.logAndDo(new RemoveFileOperation(name, currentGenerationId, purgeOnCommit));
 
 		if (purgeOnCommit) {
@@ -234,10 +254,14 @@
 		return false;
 	}
 
-
-	// @return true, if this work needs to be done on a user thread immediately
+    /**
+     * File deletion is a quick operation and typically releases substantial
+     * amount of space very quickly, this work should be done on the
+     * user thread. 
+     * @return true, this work needs to done on user thread. 
+     */
 	public boolean serviceImmediately()
 	{
-		return false;
+		return true;
 	}	
 }

Modified: db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/xact/Xact.java
URL: http://svn.apache.org/viewcvs/db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/xact/Xact.java?rev=357269&r1=357268&r2=357269&view=diff
==============================================================================
--- db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/xact/Xact.java (original)
+++ db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/xact/Xact.java Fri Dec
16 17:52:03 2005
@@ -2318,13 +2318,24 @@
 		postCompleteMode = true;
 	}
 
-	/*
-	 * Try setting the transaction to be in backup blocking state.
-	 */
-	public boolean setBackupBlockingState() {
+    /*
+     * Try setting the transaction to be in backup blocking state.
+     *
+     * @param wait if <tt>true</tt>, waits until the transaction
+     *             can be set into backup blocking state.
+     *
+     * @exception StandardException if interrupted while waiting 
+     *            for backup to complete to set the transaction into
+     *            backup blocking state.
+     */
+    public boolean setBackupBlockingState(boolean wait) 
+        throws StandardException
+    {
 		if (!inBackupBlockingState)
+        {
 			inBackupBlockingState = 
-                xactFactory.canStartBackupBlockingOperation();
+                xactFactory.canStartBackupBlockingOperation(wait);
+        }
 
 		return inBackupBlockingState;
 	}

Modified: db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/xact/XactFactory.java
URL: http://svn.apache.org/viewcvs/db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/xact/XactFactory.java?rev=357269&r1=357268&r2=357269&view=diff
==============================================================================
--- db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/xact/XactFactory.java
(original)
+++ db/derby/code/trunk/java/engine/org/apache/derby/impl/store/raw/xact/XactFactory.java
Fri Dec 16 17:52:03 2005
@@ -967,24 +967,41 @@
     }
 
 
-	/**
-	 * Checks if a backup blocking operation can be started.
-	 *
-	 * @return     <tt>true</tt> if backup blocking operations can be started.
-	 *			   <tt>false</tt> otherwise.
-	 */
-	protected boolean canStartBackupBlockingOperation()
+    /**
+     * Checks if a backup blocking operation can be started.
+     *
+     * @param wait if <tt>true</tt>, waits until a backup blocking 
+     *             operation can be started. 
+     *
+     * @return     <tt>true</tt> if backup blocking operations can be started.
+     *			   <tt>false</tt> otherwise.
+     * @exception StandardException if interrupted while waiting for backup 
+     *             to complete.
+     */
+	protected boolean canStartBackupBlockingOperation(boolean wait)
+        throws StandardException 
 	{
 		synchronized(backupSemaphore) {
-			// do not allow backup blocking operations, if online backup is
-			// is in progress.
-			if (inBackup) {
-				return false;
-			} else {
-				// not in online backup, allow backup blocking operations
-				backupBlockingOperations++;
-				return true;
+            // do not allow backup blocking operations, if online backup is
+            // is in progress.
+			if (inBackup) 
+            {
+                if(wait) {
+                    while(inBackup) {
+                        try {
+                            backupSemaphore.wait();
+                        } catch (InterruptedException ie) {
+                            throw StandardException.interrupt(ie);
+                        }
+                    }
+                }else {
+                    return false;
+                }
 			}
+
+            // not in online backup, allow backup blocking operations
+            backupBlockingOperations++;
+            return true;
 		}
 	}
 

Added: db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/master/OnlineBackupTest3.out
URL: http://svn.apache.org/viewcvs/db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/master/OnlineBackupTest3.out?rev=357269&view=auto
==============================================================================
--- db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/master/OnlineBackupTest3.out
(added)
+++ db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/master/OnlineBackupTest3.out
Fri Dec 16 17:52:03 2005
@@ -0,0 +1,40 @@
+Begin Online Backup Test3
+Initial Setup Complete
+Begin Install Jar Test
+ERROR XSRSA: Cannot backup the database when backup blocking unlogged operations are pending.
Please commit the transactions with backup blocking operations or use the backup procedure
with option to wait for them to complete. 
+Backup-1 Started
+The transaction that was blocking the backup has ended
+Backup-1 Completed
+Backup-2 Started
+Started obtest_customer.jar addition in seperate thread
+The transaction that was blocking the backup has ended
+Backup-2 Completed
+obtest_customer.jar addition is complete
+No of rows in table t1: 5
+No of rows in table customer: 3
+database shutdown properly
+Restored From the Backup
+No of rows in table t1: 5
+No of rows in table customer: 2
+database shutdown properly
+End Of Install Jar Test.
+Begin Remove Jar Test
+ERROR XSRSA: Cannot backup the database when backup blocking unlogged operations are pending.
Please commit the transactions with backup blocking operations or use the backup procedure
with option to wait for them to complete. 
+Backup-3 Started
+The transaction that was blocking the backup has ended
+Backup-3 Completed
+Backup-4 Started
+Started obtest_customer.jar remove in seperate thread
+The transaction that was blocking the backup has ended
+Backup-4 Completed
+obtest_customer.jar remove is complete
+No of rows in table t1: 8
+No of rows in table customer: 2
+database shutdown properly
+Restored From the Backup
+No of rows in table t1: 7
+No of rows in table customer: 2
+ERROR 42X96: The database class path contains an unknown jar file '"APP"."MATH_ROUTINES"'.
+database shutdown properly
+End Of Remove Jar Test.
+End Online Backup Test3

Modified: db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/suites/storemore.runall
URL: http://svn.apache.org/viewcvs/db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/suites/storemore.runall?rev=357269&r1=357268&r2=357269&view=diff
==============================================================================
--- db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/suites/storemore.runall
(original)
+++ db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/suites/storemore.runall
Fri Dec 16 17:52:03 2005
@@ -31,3 +31,4 @@
 store/OnlineCompressTest.java
 store/OnlineBackupTest1.java
 store/onlineBackupTest2.sql
+store/OnlineBackupTest3.java

Added: db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/OnlineBackupTest3.java
URL: http://svn.apache.org/viewcvs/db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/OnlineBackupTest3.java?rev=357269&view=auto
==============================================================================
--- db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/OnlineBackupTest3.java
(added)
+++ db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/OnlineBackupTest3.java
Fri Dec 16 17:52:03 2005
@@ -0,0 +1,537 @@
+/*
+
+   Derby - Class org.apache.derbyTesting.functionTests.store.OnlineBackupTest3
+
+   Copyright 2005 The Apache Software Foundation or its licensors, as applicable.
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+
+ */
+
+package org.apache.derbyTesting.functionTests.tests.store;
+import java.sql.Connection;
+import java.sql.Statement;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import org.apache.derby.tools.ij;
+import org.apache.derbyTesting.functionTests.util.TestUtil;
+import java.util.Properties;
+
+/*
+ * This class tests online backup when jar actions
+ * are running in parallel to the backup thread. 
+ *
+ * @author <a href="mailto:suresh.thalamati@gmail.com">Suresh Thalamati</a>
+ * @version 1.0
+ */
+
+public class OnlineBackupTest3 {
+
+	private static final String TEST_DATABASE_NAME = "wombat" ;
+
+	public static void main(String[] argv) throws Throwable {
+		
+        OnlineBackupTest3 test = new OnlineBackupTest3();
+   		ij.getPropertyArg(argv); 
+
+        try {
+            test.runTest();
+        }
+        catch (SQLException sqle) {
+			dumpSQLException(sqle);
+		} 
+    }
+
+
+	/*
+	 * Test online backup with unlogged jar operations running in parallel. 
+	 */
+	private void runTest() throws SQLException, Exception {
+		logMessage("Begin Online Backup Test3");
+		Connection conn = ij.startJBMS();
+        conn.setAutoCommit(false);
+        Statement stmt = conn.createStatement();
+        stmt.execute("create table t1(a int ) ");
+        stmt.execute("insert into t1 values(1)");
+        stmt.execute("insert into t1 values(2)");
+        stmt.execute("create table customer(id int , name varchar(100))");
+        stmt.execute("insert into customer values(1, 'ABC')");
+        stmt.execute("insert into customer values(2, 'XYZ')");
+        String crproc = "create procedure addCustomer(id INT, name VARCHAR(100)) " +
+            "MODIFIES SQL DATA " + 
+            "external name " + 
+            "'org.apache.derbyTesting.backupRestore.Customer.addCustomer' " + 
+            " language java parameter style java ";
+            
+        stmt.execute(crproc);
+
+        String dvfunc = "create function dv(P1 INT) RETURNS INT NO SQL " +
+            " external name 'dbytesting.CodeInAJar.doubleMe' " + 
+            " language java parameter style java " ;
+
+        stmt.execute(dvfunc) ;
+        conn.commit();
+        
+        logMessage("Initial Setup Complete");
+
+        // perform install jar operation with 
+        // online backup running in parallel.
+        installJarTest();
+
+        // perform remove jar operation with 
+        // online backup running in parallel.
+        removeJarTest();
+
+		logMessage("End Online Backup Test3");
+	}
+
+		
+	/**
+	 * Shutdown the datbase
+	 * @param  dbName  Name of the database to shutdown.
+	 */
+	void shutdown(String dbName) {
+
+		try{
+			// shutdown 
+            TestUtil.shutdownUsingDataSource(TEST_DATABASE_NAME);
+		}catch(SQLException se){
+			if (se.getSQLState() != null && se.getSQLState().equals("08006"))
+				System.out.println("database shutdown properly");
+			else
+				dumpSQLException(se);
+		}
+	}
+
+    /*
+     * get connection to the test database
+     */
+    Connection getConnection() throws SQLException 
+    {
+        Properties prop = new Properties();
+        prop.setProperty("databaseName", TEST_DATABASE_NAME);
+        Connection conn = TestUtil.getDataSourceConnection(prop);
+        return conn;
+    }
+
+
+	/**
+	 * Write message to the standard output.
+	 */
+	void logMessage(String   str)	{
+			System.out.println(str);
+	}
+
+	
+	/**
+	 * dump the SQLException to the standard output.
+	 */
+	static private void dumpSQLException(SQLException sqle) {
+		
+		org.apache.derby.tools.JDBCDisplayUtil.	ShowSQLException(System.out, sqle);
+		sqle.printStackTrace(System.out);
+	}
+
+    
+    private int countRows(Connection conn, 
+                          String tableName) 
+        throws SQLException
+    {
+        Statement s = conn.createStatement();
+        ResultSet rs = s.executeQuery("SELECT count(*) from " +  tableName );
+        rs.next();
+        int noRows = rs.getInt(1);
+        rs.close();
+        s.close();
+        return noRows;
+    }
+
+    /*
+     * Test install jar running in parallel to backup and vice versa. 
+     */
+    void installJarTest() throws SQLException, Exception{
+        logMessage("Begin Install Jar Test");
+        Connection conn1 = getConnection();
+        conn1.setAutoCommit(false);
+        Statement conn1_stmt = conn1.createStatement();
+        Connection conn2 = getConnection();
+        conn2.setAutoCommit(false);
+        Statement conn2_stmt = conn2.createStatement();
+
+        
+        conn1_stmt.execute(
+           "call sqlj.install_jar('extin/brtestjar.jar', 'math_routines', 0)");
+        
+        try {
+            // followng backup call should because jar operation is pending 
+           conn2_stmt.execute(
+            "call SYSCS_UTIL.SYSCS_ONLINE_BACKUP_DATABASE('extinout/mybackup', 0)");
+        } catch (SQLException sqle) {
+            //above statement should have failed. 
+            org.apache.derby.tools.JDBCDisplayUtil.	ShowSQLException(System.out, sqle);
+        }
+
+        // invoke backup in another thread, should block for the above install jar 
+        // operation for 'brtestjar.jar to commit.
+        
+        // start a  thread to perform online backup
+		OnlineBackup backup = new OnlineBackup(TEST_DATABASE_NAME);
+		Thread backupThread = new Thread(backup, "BACKUP1");
+		backupThread.start();	
+		// wait for the backup to start
+		backup.waitForBackupToBegin();
+		logMessage("Backup-1 Started");
+
+        // sleep for few seconds just to make sure backup thread is actually
+		// gone to a wait state for unlogged actions to commit.
+		java.lang.Thread.sleep(1000);
+			
+		// backup should not even start doing real work before the
+		// unlogged transaction is commited
+		if(!backup.isRunning())
+			logMessage("Backup is not waiting for unlogged " +  
+                       "install jar action to commit");
+
+        //insert some rows that should appear in the backup.
+        conn1_stmt.execute("insert into t1 values(3)");
+        conn1_stmt.execute("insert into t1 values(4)");
+
+        
+        // set the database class with both the jars  installed above.
+        conn1_stmt.execute("CALL SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY( " + 
+                           "'derby.database.classpath', " + 
+                           "'APP.math_routines') " ) ;
+
+        //Now commit the jar operation in connection1 for backup to proceed. 
+        conn1_stmt.execute("insert into t1 values(5)");
+        //commit the transaction with jar opearation that is blocking the backup.
+        conn1.commit();
+        logMessage("The transaction that was blocking the backup has ended");
+
+        // wait for backup to finish. 
+        backup.waitForBackupToEnd();
+		backupThread.join();
+        logMessage("Backup-1 Completed");
+        
+        // Case : jar op should block if backup is in progress
+        // add a index that will block the backup until it is committted.
+        conn1_stmt.execute("create index idx1 on customer(id)");
+        conn1_stmt.execute("insert into t1 values(6)");
+        
+        // start a  thread to perform online backup
+        backup = new OnlineBackup(TEST_DATABASE_NAME);
+        backupThread = new Thread(backup, "BACKUP2");
+		backupThread.start();	
+		// wait for the backup to start
+		backup.waitForBackupToBegin();
+		logMessage("Backup-2 Started");
+
+        // sleep for few seconds just to make sure backup thread is actually
+		// gone to a wait state for unlogged actions to commit.
+		java.lang.Thread.sleep(1000);
+			
+		// backup should not even start doing real work before the
+		// unlogged transaction is commited
+		if(!backup.isRunning())
+			logMessage("Backup is not waiting for unlogged " +  
+                       "index action to commit");
+
+
+        // add another jar file  , this one should block and 
+        // should not get into the backup. Backup does not allow new 
+        // jar operation if it is already waiting for backup blocking
+        // to complete(commit/rollback). 
+
+        AsyncStatementThread asyncJarActionThread = 
+            new AsyncStatementThread(conn2, 
+          "call sqlj.install_jar('extin/obtest_customer.jar', 'customer_app', 0)");
+        asyncJarActionThread.start();
+        logMessage("Started obtest_customer.jar addition in seperate thread");
+
+        //sleep for few seconds to give a chance for the 
+        //jar addition thread to get into action.
+        java.lang.Thread.sleep(1000);
+
+        //roll back the index op. Backup should proceed now.
+        conn1.rollback();
+        logMessage("The transaction that was blocking the backup has ended");
+
+        // wait for backup to finish. 
+        backup.waitForBackupToEnd();
+		backupThread.join();
+        logMessage("Backup-2 Completed");
+        
+        // wait for customer app jar installation to finish now. 
+        asyncJarActionThread.join();
+        logMessage("obtest_customer.jar addition is complete");
+
+        // set the database class with both the jars  installed above.
+        conn1_stmt.execute("CALL SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY( " + 
+                           "'derby.database.classpath', " + 
+                           "'APP.customer_app:APP.math_routines') " ) ;
+        
+        conn1.commit();
+
+        // second jar must have got installed after the backup. 
+        // call a function in the custome_app jar
+        conn1_stmt.execute("call addCustomer(3 , 'John')");
+        conn1.commit();
+        
+        logMessage("No of rows in table t1: " + countRows(conn1, "T1"));
+        logMessage("No of rows in table customer: " + 
+                   countRows(conn1, "customer"));
+        conn1.commit();
+        conn2.commit();
+        conn1_stmt.close();
+        conn2_stmt.close();
+        conn1.close();
+        conn2.close();
+        
+        //shutdown the test db 
+		shutdown(TEST_DATABASE_NAME);
+		// restore the database from the backup and run some checks 
+		backup.restoreFromBackup();
+		logMessage("Restored From the Backup");
+        Connection conn = getConnection();
+        Statement stmt = conn.createStatement();
+        logMessage("No of rows in table t1: " + countRows(conn, "T1"));
+        logMessage("No of rows in table customer: " + 
+                   countRows(conn, "customer"));
+        // execute select statement using the "dv" funciont.  
+        stmt.execute("select dv(a) from t1");
+
+        
+        try {
+            // set the database class with both the jars  installed above.
+            stmt.execute("CALL SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY( " + 
+                           "'derby.database.classpath', " + 
+                           "'APP.customer_app:APP.math_routines') " ) ;
+            stmt.execute("call addCustomer(3 , 'John')"); 
+        }catch(SQLException se) {
+            //ignore for now. No sure way to 
+            //check that jar did not get into backup 
+            //without debug flags. 
+        }
+
+        stmt.close();
+        conn.close();
+
+		//shutdown the test db 
+		shutdown(TEST_DATABASE_NAME);
+		logMessage("End Of Install Jar Test.");
+
+    }
+
+
+    /*
+     * Test remove jar running in parallel to backup and vice versa. 
+     */
+    void removeJarTest() throws SQLException, Exception{
+        logMessage("Begin Remove Jar Test");
+        Connection conn1 = getConnection();
+        conn1.setAutoCommit(false);
+        Statement conn1_stmt = conn1.createStatement();
+        Connection conn2 = getConnection();
+        conn2.setAutoCommit(false);
+        Statement conn2_stmt = conn2.createStatement();
+        try {
+            conn1_stmt.execute(
+           "call sqlj.install_jar('extin/obtest_customer.jar', 'customer_app', 0)");
+        }catch(SQLException se) {
+            //it is ok if was jar already there.
+        }
+
+        // remove both the jars from the class path , 
+        // so that we can remove them. 
+        conn1_stmt.execute("CALL SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY( " + 
+                        "'derby.database.classpath', '')") ;
+        conn1.commit();
+
+        conn1_stmt.execute(
+           "call sqlj.remove_jar('APP.math_routines', 0)");
+        
+        try {
+            // followng backup call should because remove 
+            // jar operation is pending 
+           conn2_stmt.execute(
+            "call SYSCS_UTIL.SYSCS_ONLINE_BACKUP_DATABASE('extinout/mybackup', 0)");
+        } catch (SQLException sqle) {
+            //above statement should have failed. 
+            org.apache.derby.tools.JDBCDisplayUtil.	ShowSQLException(System.out, sqle);
+        }
+            
+        // invoke backup in another thread, should block for 
+        // the above remove jar  to commit.
+        
+        // start a  thread to perform online backup
+		OnlineBackup backup = new OnlineBackup(TEST_DATABASE_NAME);
+		Thread backupThread = new Thread(backup, "BACKUP3");
+		backupThread.start();	
+		// wait for the backup to start
+		backup.waitForBackupToBegin();
+		logMessage("Backup-3 Started");
+
+        // sleep for few seconds just to make sure backup thread is actually
+		// gone to a wait state for unlogged actions to commit.
+		java.lang.Thread.sleep(1000);
+			
+		// backup should not even start doing real work before the
+		// unlogged transaction is commited
+		if(!backup.isRunning())
+			logMessage("Backup is not waiting for unlogged " +  
+                       "remove jar action to commit");
+
+        //insert some rows that should appear in the backup.
+        conn1_stmt.execute("insert into t1 values(10)");
+        conn1_stmt.execute("insert into t1 values(11)");
+
+        //commit the transaction with jar opearation that is blocking the backup.
+        conn1.commit();
+        logMessage("The transaction that was blocking the backup has ended");
+        
+        // wait for backup to finish. 
+        backup.waitForBackupToEnd();
+		backupThread.join();
+
+        logMessage("Backup-3 Completed");
+        
+        // Case 2: remove jar op should block if backup is in progress
+        // add a index that will block the backup until it is committted.
+        conn1_stmt.execute("create index idx1 on customer(id)");
+        conn1_stmt.execute("insert into t1 values(12)");
+        
+        // start a  thread to perform online backup
+        backup = new OnlineBackup(TEST_DATABASE_NAME);
+        backupThread = new Thread(backup, "BACKUP4");
+		backupThread.start();	
+		// wait for the backup to start
+		backup.waitForBackupToBegin();
+		logMessage("Backup-4 Started");
+
+        // sleep for few seconds just to make sure backup thread is actually
+		// gone to a wait state for unlogged actions to commit.
+		java.lang.Thread.sleep(1000);
+			
+		// backup should not even start doing real work before the
+		// unlogged transaction is commited
+		if(!backup.isRunning())
+			logMessage("Backup is not waiting for unlogged " +  
+                       "index action to commit");
+
+
+        // remove another jar file  , this one should block and 
+        // should not get into the backup. Backup does not allow new 
+        // jar operation if it is already waiting for backup blocking
+        // to complete(commit/rollback). 
+
+        AsyncStatementThread asyncJarActionThread = 
+            new AsyncStatementThread(conn2, 
+          "call sqlj.remove_jar('APP.customer_app', 0)");
+        asyncJarActionThread.start();
+        logMessage("Started obtest_customer.jar remove in seperate thread");
+
+        //sleep for few seconds to give a chance for the 
+        //jar addition thread to get into action.
+        java.lang.Thread.sleep(1000);
+
+        //roll back the index op. Backup should proceed now.
+        conn1.rollback();
+        logMessage("The transaction that was blocking the backup has ended");
+        // wait for backup to finish. 
+        backup.waitForBackupToEnd();
+		backupThread.join();
+        logMessage("Backup-4 Completed");
+
+        // wait for customer app jar installation to finish now. 
+        asyncJarActionThread.join();
+        logMessage("obtest_customer.jar remove is complete");
+        
+        //this insert should not apprear on restore.
+        conn1_stmt.execute("insert into t1 values(13)");
+
+        logMessage("No of rows in table t1: " + countRows(conn1, "T1"));
+        logMessage("No of rows in table customer: " + 
+                   countRows(conn1, "customer"));
+        conn1.commit();
+        conn2.commit();
+        conn1_stmt.close();
+        conn2_stmt.close();
+        conn1.close();
+        conn2.close();
+        
+        //shutdown the test db 
+		shutdown(TEST_DATABASE_NAME);
+		// restore the database from the backup and run some checks 
+		backup.restoreFromBackup();
+		logMessage("Restored From the Backup");
+        Connection conn = getConnection();
+        Statement stmt = conn.createStatement();
+        logMessage("No of rows in table t1: " + countRows(conn, "T1"));
+        logMessage("No of rows in table customer: " + 
+                   countRows(conn, "customer"));
+
+        // check if the jar removal was successful.
+        // APP.math_routines should not be in backup.
+        try {
+            // set the database class with both the jars  installed above.
+            stmt.execute("CALL SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY( " + 
+                           "'derby.database.classpath', " + 
+                           "'APP.math_routines') " ) ;
+        }catch (SQLException sqle) {
+            //above statement should have failed. 
+            org.apache.derby.tools.JDBCDisplayUtil.	ShowSQLException(System.out, sqle);
+        }
+        
+
+        stmt.close();
+        conn.close();
+
+		//shutdown the test db 
+		shutdown(TEST_DATABASE_NAME);
+		logMessage("End Of Remove Jar Test.");
+
+    }
+
+
+    /*
+     * Run a sql statement in a seperate thread. 
+     */
+    class AsyncStatementThread extends Thread {
+        Connection conn;
+        String stmt;
+    
+        AsyncStatementThread(Connection conn, String stmt) {
+            this.conn = conn;
+            this.stmt = stmt;
+        }
+
+        public void run() {
+            Statement aStatement = null;
+            try {
+                aStatement = conn.createStatement();
+                aStatement.execute(stmt);
+                aStatement.close();
+                // commit here, it is possible that 
+                // this thread may have got into action 
+                // before the backup went into wait state.
+                conn.commit();
+            } catch (SQLException sqle) {
+                org.apache.derby.tools.JDBCDisplayUtil.ShowSQLException(System.out, sqle);
+				sqle.printStackTrace(System.out);
+            }
+            aStatement = null;
+        }
+    }
+
+}

Added: db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/OnlineBackupTest3_app.properties
URL: http://svn.apache.org/viewcvs/db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/OnlineBackupTest3_app.properties?rev=357269&view=auto
==============================================================================
--- db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/OnlineBackupTest3_app.properties
(added)
+++ db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/OnlineBackupTest3_app.properties
Fri Dec 16 17:52:03 2005
@@ -0,0 +1,22 @@
+#
+# *** DO NOT PUT PROPERTIES FOR THE DERBY SYSTEM IN THIS FILE.  THEY BELONG
+# *** IN the _derby.properties file.
+#
+# It will get handed to the test on the command line in a -p <filename>
+# argument.
+#
+# This causes ij (or the GUI on ij) to load the driver and make an
+# initial connection to the database.
+#
+#
+
+#database=jdbc:derby:wombat;create=true;logDevice=extinout/br1logDir
+usedefaults=true
+useextdirs=true
+supportfiles=tests/store/brtestjar.jar,tests/store/obtest_customer.jar
+
+#Exclude for J2ME/Foundation - test requires java.sql.DriverManager
+runwithfoundation=false
+
+#exclude with SecurityManager DERBY-709
+noSecurityManager=true

Modified: db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/copyfiles.ant
URL: http://svn.apache.org/viewcvs/db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/copyfiles.ant?rev=357269&r1=357268&r2=357269&view=diff
==============================================================================
--- db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/copyfiles.ant
(original)
+++ db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/copyfiles.ant
Fri Dec 16 17:52:03 2005
@@ -144,3 +144,5 @@
 OnlineBackupTest1_app.properties
 onlineBackupTest2.sql
 onlineBackupTest2_app.properties
+OnlineBackupTest3_app.properties
+obtest_customer.jar

Added: db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/obtest_customer.jar
URL: http://svn.apache.org/viewcvs/db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/obtest_customer.jar?rev=357269&view=auto
==============================================================================
Binary file - no diff available.

Propchange: db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/store/obtest_customer.jar
------------------------------------------------------------------------------
    svn:mime-type = application/octet-stream



Mime
View raw message