db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Van Couvering <David.Vancouver...@Sun.COM>
Subject Re: [PATCH] Derby 218 - Add relaxed durability option
Date Mon, 02 May 2005 17:14:18 GMT
After I wrote my email, I wondered if the feature as currently 
implemented wasn't quite what I was thinking we should have.  I agree 
that if you potentially can't boot your database, this is not a 
production-level feature.

I second the motion to see more work done to implement the relaxed 
durability option that guarantees a bootable database, that's what I was 
thinking of when I sent my previous email.

Thanks,

David

Mike Matrigali wrote:

> As I have posted before I have concerns about this mode, as it can
> definitely produce a database which can not boot.  But the discussions
> here convinced me
> that it was useful for a number of applications, so now believe it
> should be checked in.  I definitely have no career path in marketing
> as my first take at a name was corrupt_your_db=true :-) .  So testMode
> seemed ok to me, but if another name is better that is fine with me
> also.
> 
> Going forward I would like to see more work done to implement the
> relaxed durability option suresh and I were discussing.  I believe
> that with some code changes file allocation, log write and many of
> the data write sync's can be eliminated and still guarantee a bootable
> database.  This mode would not guarantee a committed transaction,
> but would guarantee that database on a disk with enough free space
> would boot and that the db would represent only complete transactions 
> (either fully applied or not).  This option would
> be slower than what is available from this patch so I believe both
> features are useful.
> 
> David Van Couvering wrote:
> 
>> Hi, Sunitha.  Thanks for your work on this.  My comment is more a 
>> "marketing" comment than a technical one.  I disagree that this is 
>> useful only for testing and development.  As I mentioned in previous 
>> emails, there are a number of customers who may find this very useful 
>> indeed, who are willing to trade off 100% guarantee of all 
>> transactions being available on recovery for significant increases in 
>> performance.
>>
>> I wouldn't get into this debate were it not for the fact that the 
>> property you have created is called "testMode."  I would much rather 
>> have it not imply it's only for testing, with a name like 
>> "delayedWrite" or "relaxedDurability" or some such.
>>
>> Thanks,
>>
>> David
>>
>> Sunitha Kambhampati wrote:
>>
>>> A little background: Sometime earlier on the list, Dan posted a fix 
>>> to make derby go faster with relaxed durability with some flags.  The 
>>> post is at 
>>> http://article.gmane.org/gmane.comp.apache.db.derby.user/681/match=relaxed+durability 
>>>
>>> This mode is very useful for unit testing or at development time when 
>>> recoverability is not required.
>>> Basically in this mode, syncs are disabled for logs, data writes at 
>>> checkpoint, page allocation when file is grown; - what this means is 
>>> that data is not flushed all the way to the disk and in most cases 
>>> i/o cost is not involved. Note,  code already exists in Derby to do 
>>> this.
>>> So for Derby 218, This  patch addresses the following requirements:  
>>> 1) Have a single property to enable this relaxed durability mode, so 
>>> it is easy for users  to enable it.
>>> 2) Print message to derby.log that this mode is enabled
>>> 3) A way to report boot errors that may be because of this option 
>>> being enabled. What this maps to is - have a marker to recognize that 
>>> database was booted in this mode, and then on subsequent boot; if 
>>> errors happen during recovery - report a message that it could have 
>>> happened because of this mode being set.
>>>
>>> Changes are not much.  svn stat
>>> M      java\engine\org\apache\derby\impl\store\raw\log\LogToFile.java
>>> M      
>>> java\engine\org\apache\derby\impl\store\raw\data\BaseDataFileFactory.java 
>>>
>>> M      
>>> java\engine\org\apache\derby\impl\store\raw\data\RAFContainer.java
>>> M      java\engine\org\apache\derby\iapi\reference\MessageId.java
>>> M      java\engine\org\apache\derby\iapi\reference\Property.java
>>> M      java\engine\org\apache\derby\loc\messages_en.properties
>>> A      
>>> java\testing\org\apache\derbyTesting\functionTests\tests\store\TestNoSyncMode.java 
>>>
>>> M      
>>> java\testing\org\apache\derbyTesting\functionTests\tests\store\copyfiles.ant 
>>>
>>> A      
>>> java\testing\org\apache\derbyTesting\functionTests\tests\store\TestNoSyncMode_derby.properties 
>>>
>>> A      
>>> java\testing\org\apache\derbyTesting\functionTests\master\TestNoSyncMode.out 
>>>
>>> M      
>>> java\testing\org\apache\derbyTesting\functionTests\suites\storemats.runall 
>>>
>>>
>>> Since this is a longish email , for those who may not  read the rest 
>>> of it, here are the testing results
>>>
>>> Testing results:
>>> - I ran derbyall with 1.4.2 jvm and all tests passed.     - I ran the 
>>> storeall suite  separately both with setting the 
>>> derby.system.testMode property to true and without it.     with this 
>>> property set, the storeall suite ran about 2 times as fast.
>>> -------------------------------
>>>
>>> Below I explain how the patch addresses the above requirements  and 
>>> try to provide details on how I went about making those changes
>>>
>>> 0) Note
>>> 1) A single property has been added to enable this relaxed durability 
>>> option.  The property name is derby.system.testMode and takes in 
>>> value true or false. By default it is false. It can be set as a 
>>> command line option to the JVM when starting the application or 
>>> within the application or in the derby.properites file.  Property is 
>>> loaded at boot time, so for it to have an effect you need to enable 
>>> it at boot time.  More comments added in code itself. see variable 
>>> TESTMODE_NO_SYNC =  "derby.system.testMode"; in Property.java.
>>>
>>> (In this patch, I have used 'no_sync' as suffix  in variable names 
>>> because I think with respect to store - that is what this mode is all 
>>> about - disabling syncs for the cases already mentioned earlier in 
>>> this mail)
>>>
>>> I chose the property name derby.system.testMode for this
>>> - represents a higher concept as to when one would use  relaxed 
>>> durability (no sync) option.
>>> - mostly it should be enabled only when data is not important and 
>>> ideally maybe for testing and development purposes,
>>> - also with the hope that the word 'testMode'  would prompt users to 
>>> look this property up before using it in a production like 
>>> environment where it may not necessarily be appropriate.
>>>
>>> Code changes for this:
>>> - Added this new property  derby.system.testMode in
>>> java\engine\org\apache\derby\iapi\reference\Property.java
>>>
>>> - Removed the following properties from Property.java as they are no 
>>> longer required.
>>>    derby.storage.dataNotSyncedAtCheckPoint=true    ( I'll refer to 
>>> this as DNSAC   in notes below)
>>>    derby.storage.dataNotSyncedAtAllocation=true       ( I'll refer to 
>>> this as DNSAA  in notes below)
>>>    derby.storage.logNotSynced=true            (I'll refer to this as 
>>> LNS in notes below)
>>>
>>> - Previously code existed for enabling  the DNSAC, DNSAA and LNS 
>>> options conditional on Performance.MEASURE
>>>
>>> So how it works is , the 2 variables in BaseDataFileFactory.java are 
>>> used to control if syncs happen or not for data pages at allocation 
>>> and at checkpoint.
>>> dataNotSyncedAtAllocation  dataNotSyncedAtCheckpoint
>>>
>>> And  in java/engine/org/apache/derby/impl/store/raw/log/LogToFile, 
>>> the logNotSynced variable is used to control if syncs happen for the 
>>> logs are not.
>>>
>>> These variables are enabled (set to true) if Performance.MEASURE is 
>>> set to true, and if the DNSAC, and DSNAA propertie are enabled.
>>>
>>> So all that is required now is
>>> - remove conditional on Performance.MEASURE in these cases, instead 
>>> use the derby.system.testMode property to enable them . This happens 
>>> in the boot methods
>>> - make sure to remove any conditionals on Performance.MEASURE that is 
>>> used to disable syncs but instead control it by the variables 
>>> mentioned above.. ie e.g. dataNotSyncedAtAllocation in 
>>> BaseDataFileFactory.java ...
>>> - remove reading of the DNSAC, DNSAA,LNS property as they are not 
>>> needed anymore
>>>
>>> Changes were needed to following files -  i) 
>>> java/engine/org/apache/derby/impl/store/raw/data/BaseDataFileFactory.java  
>>> (check boot method)
>>> ii) java/engine/org/apache/derby/impl/store/raw/data/RAFContainer.java
>>> iii) java/engine/org/apache/derby/impl/store/raw/log/LogToFile.java   
>>> (check boot method).
>>>
>>> 2)  Write to the derby.log a warning message
>>>
>>> Needed to add a new message.
>>> Messages need to be translated, so I made the following changes
>>>
>>> i) added a variable  in 
>>> java/engine/org/apache/derby/iapi/reference/MessageId.java  to 
>>> represent this new message.
>>>  String STORE_TESTMODE_NO_SYNC           = "D013";
>>>
>>> ii) Message content added to 
>>> java/engine/org/apache/derby/loc/messages_en.properties
>>> D013=WARNING: {0} is set to true. In this mode, there is no guarantee 
>>> that the system will recover a system crash or JVM termination. 
>>> Enable this property only when there is no need for recoverability \n
>>>
>>> iii) In the boot process, rawstore boot calls logfactory boot, so the 
>>> first time the derby.system.testMode property is read, it is in 
>>> rawstore, so added code to print the warning message in 
>>> BaseDataFileFactory.boot(..)
>>>
>>>
>>> 3)  A way to report boot errors that may be because of this option 
>>> being enabled. What this maps to is - have a marker to recognize that 
>>> database was booted in this mode, and then on subsequent boot if 
>>> errors happen during recovery report a message that it could have 
>>> happened because of this mode being set.
>>>
>>> - To write a marker out , log.ctrl file is used, it has a few spare 
>>> bytes so no major changes needed here. ( ie no disk format changes.... )
>>> Check the method readControlFile  in  
>>> java/engine/org/apache/derby/impl/store/raw/log/LogToFile.java
>>>
>>> - Used the first spare byte available in log control file which will 
>>> be used to store information about if derby.system.testMode is set or 
>>> not.
>>> Added variable TESTMODE_NO_SYNC_FLAG_OFFSET to represent this offset 
>>> in LogToFile.java
>>>
>>> - The value written out if this property is enabled is  
>>> TESTMODE_NO_SYNC_TRUE = 0x1 else a 0 is written out.  ( I didnt like 
>>> to put the 0 but wasnt feeling too strong to adding a new static byte 
>>> variable either.. and  as rest of the code around it in 
>>> writeControlFile seems to use 0 as is , I just followed the same 
>>> approach).
>>>
>>> -  Change made to readControlFile to read in the value for the 
>>> testmode case when Logfactory is booted, the flag is read and value 
>>> stored
>>> in  variable called wasDBInTestModeNoSync. This variable keeps track 
>>> of if database was previously booted in the derby.system.testMode or 
>>> not.  Comes in use for error reporting.
>>>
>>> - Also added a message to report to the derby.log incase database was 
>>> previously booted with derby.system.testMode set to true. Change to 
>>> messages_en.properties and MessageId.java.
>>> L020=WARNING: The database was previously booted with {0} set to 
>>> true. Note that this mode {0}=true does not guarantee recoverability \n
>>>
>>> little on the flow with respect to writeControlFile .. if you look at 
>>> the code in LogToFile for boot and recover, writeControlFile is 
>>> called a couple of times.. to read the checkpoint instant etc..
>>> - For my changes, I am just copy/paste the comments from 
>>> writeControlFile(...)  , Note case4 below .. which is the interesting 
>>> one.
>>>
>>>       // A spare byte at offset TESTMODE_NO_SYNC_FLAG_OFFSET
>>>        // is used to write out if database was in 
>>> derby.system.testMode=true mode
>>>        // or not.
>>>        // This byte will have one of the two values
>>>        // - TESTMODE_NO_SYNC_TRUE to represent database was booted
>>>        //   in this mode
>>>        // -  0 to represent database was not booted in this mode
>>>        //         // The sole purpose of this flag in control file is 
>>> to help
>>>        // identify any recovery problems that might happen as a 
>>> result of this
>>>        // mode being set.
>>>        //
>>>        // Possible scenarios. In table below, cases are listed giving
>>>        // the value of derby.system.testMode if it was enabled or not 
>>> at boot time
>>>        // 
>>> ---------------------------------------------------------------
>>>        //  Case   Previous       Current         Action , what should 
>>> the flag be set to in log.ctrl
>>>        //               Boot           Boot               file
>>>        // --------------------------------------------------------------
>>>        //   1       -             true                write 
>>> TESTMODE_NO_SYNC_TRUE
>>>        //   2       -                 false           write 0
>>>        //   3       false        false               write 0
>>>        //   4       true          false               write 
>>> TESTMODE_NO_SYNC_TRUE
>>>        //   5       true          true                write 
>>> TESTMODE_NO_SYNC_TRUE
>>>        //
>>>        // Case 4 is the interesting case here.
>>>        // one cannot write out the derby.system.testMode is disabled 
>>> just yet
>>>        // because it is possible that the database recovery might 
>>> fail and on subsequent
>>>        // restarts this state would be lost. So only when recovery is 
>>> successful, will
>>>        // we update the flag in the log control file to indicate that 
>>> the derby.system.testMode
>>>        // is not enabled.  (This will happen at the end of recovery , 
>>> see recover method)
>>>        //
>>>        // Case 1, 2 is when database is created fresh with this 
>>> property enabled.
>>>        //
>>>        // wasDBInTestModeNoSync has information if database was 
>>> previously booted
>>>        // in derby.system.testMode=true mode
>>>
>>> To handle case 4, changes went into recover () method , if database 
>>> recovery is successful only then we update the flag in the control 
>>> file to say database is not booted with this property enabled.  Since 
>>> there was no method available to update the log control file, added a 
>>> new method updateTestModeFlagInCtrlFile(byte value) to do that.
>>>
>>> So now that changes are done to write the marker to the control file 
>>> and to read it, it is necessary to take care of case when error happens.
>>>
>>> - added a new message  to report that cause of an error could be 
>>> related to this property being set
>>> added a new variable in  MessageId.LOG_TESTMODE_NO_SYNC_ERR in   
>>> java/engine/org/apache/derby/iapi/reference/MessageId.java
>>> and the message to 
>>> java/engine/org/apache/derby/loc/messages_en.properties
>>> L021=WARNING: The database was either previously booted or is 
>>> currently booted with {0} set to true. Note that this mode {0}=true 
>>> does not guarantee recoverability, and is probably what might have 
>>> caused this boot error \n
>>> - added  MessageId.LOG_TESTMODE_NO_SYNC_ERR in   
>>> java/engine/org/apache/derby/iapi/reference/MessageId.java
>>>
>>> Added a new method logErrMsgForTestModeNoSync() in LogToFile.java to 
>>> print out this message if any error happens when database is in the 
>>> influence of this property.
>>>
>>> In deciding what the best place would be to call the 
>>> logErrMsgForTestModeNoSync, I debated between adding it in 
>>> markCorrupt() versus logErrMsg(..).  It seemed like if error happened 
>>> during boot - the flow was to call markCorrupt which would call 
>>> logErrMsg.  Some error messages seem to call logErrMsg(..) 
>>> directly.   So I picked the logErrMsg as it covered most of the 
>>> errors thrown during boot/recovery that would be related to something 
>>> with this property being set.  
>>> -------------------------------------------
>>>
>>> TESTING
>>>
>>> 1) I added one new  java file  per the derby test harness,   to test 
>>> the following
>>>  -  check if the property is enabled correctly by timing some inserts 
>>> into a table.
>>>  -  check if the correct flag (TESTMODE_NO_SYNC_TRUE) is written out 
>>> into the log.ctrl file
>>>  -  check if the derby.log has the warning message.
>>>
>>> Changes include
>>> i)Added new test to   
>>> java/testing/org/apache/derbyTesting/functionTests/tests/store/TestNoSyncMode.java 
>>>
>>> ii)Added 
>>> java/testing/org/apache/derbyTesting/functionTests/tests/store/TestNoSyncMode_derby.properties 
>>>
>>>
>>> TestNoSyncMode_derby.properties has
>>> derby.system.testMode=true
>>>
>>> This property file is needed so when test is run it picks up this 
>>> property.
>>>
>>> iii)The TestNoSyncMode_derby.properties is added to copyfiles.ant so 
>>> it will be copied over when compiling derby tests.
>>>
>>> iv) To enable this new  test as part of the store suite, I modified the
>>> java/testing/org/apache/derbyTesting/functionTests/suites/storemats.runall 
>>>
>>> added store/TestNoSyncMode.java
>>>
>>> v) To get my correct output file, first i just put an initial empty 
>>> file in 
>>> java/testing/org/apache/derbyTesting/functionTests/master/TestNoSyncMode.out 
>>>
>>>
>>> Ran the test  after compiling source;  java 
>>> org.apache.derbyTesting.functionTests.harness.RunTest 
>>> store/TestNoSyncMode.java
>>> Then I updated the out file in the master directory, after checking 
>>> if the TestNoSyncMode.out file was correct.
>>>
>>> 2) For the error messages on a boot error, it is difficult to put it 
>>> in a test, so I checked this in the debugger and it works ok.
>>>
>>> Meanwhile, Suresh Thalamati posted on the list that he is working on 
>>> a corruptible storage factory, once that is there - I am hoping I 
>>> could use that to have a test to check the error messages for boot 
>>> errors.
>>>
>>> As far as I can tell, this does not affect upgrade and such since we 
>>> are not changing the disk format etc.. ..There is one spare byte  in 
>>> log.ctrl file that is being used now as a result of this change  and 
>>> this byte value should not have been used to represent anything 
>>> useful by any earlier version
>>>
>>> Testing results (already  mentioned in the beginning of this email).
>>>
>>> Please give your comments. Thanks,
>>> Sunitha.
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> Index: java/engine/org/apache/derby/impl/store/raw/log/LogToFile.java
>>> ===================================================================
>>> --- java/engine/org/apache/derby/impl/store/raw/log/LogToFile.java    
>>> (revision 165265)
>>> +++ java/engine/org/apache/derby/impl/store/raw/log/LogToFile.java    
>>> (working copy)
>>> @@ -265,7 +265,38 @@
>>>  
>>>      /* Log Control file flags. */
>>>      private static final byte IS_BETA_FLAG = 0x1;
>>> -
>>> +   +    /**
>>> +     * When the derby.system.testMode property is enabled, the store 
>>> +     * system will not force sync calls in the following cases
>>> +     * - for the log file at each commit
>>> +     * - for the log file before data page is forced to disk
>>> +     * - for page allocation when file is grown
>>> +     * - for data writes during checkpoint
>>> +     * This means the recovery system may not work properly
>>> +     * In order that we recognize this case that the database was 
>>> previously
>>> +     * booted in this mode, this value is written out
>>> +     * into the log control file. This will help prevent us from 
>>> +     * wasting time to resolve issues in such cases. +     * @see 
>>> org.apache.derby.iapi.reference.Property#TESTMODE_NO_SYNC
>>> +     */
>>> +    private static final byte TESTMODE_NO_SYNC_TRUE = 0x1;
>>> +   +    /**
>>> +     * A byte in the log control file is used to store information
>>> +     * about if the database is booted in derby.system.testMode mode
>>> +     * The first available spare byte that is available for this 
>>> purpose is +     * after 5 ints ( 4bytes each), a long ( 8 bytes) and 
>>> 1 byte in the log control
>>> +     * file. Hence the offset is calculated as follows
>>> +     */
>>> +    private static final int TESTMODE_NO_SYNC_FLAG_OFFSET =  5*4 + 8 
>>> + 1;
>>> +    +    /**
>>> +     * keeps track of if the database was booted previously with 
>>> +     * derby.system.testMode=true +     */
>>> +    private static boolean wasDBInTestModeNoSync = false;
>>> +         /* to err on the conservative side, unless otherwise set, 
>>> assume log
>>>       *    archive is ON       */
>>> @@ -362,7 +393,7 @@
>>>      // log goes to the log subdirectory underneath the data directory
>>>      String logDevice;
>>>  
>>> -    // debug only flag - disable syncing of log file for debugging 
>>> performance.
>>> +    // disable syncing of log file when running in 
>>> derby.system.testMode enabled
>>>      private boolean logNotSynced = false;
>>>  
>>>      private boolean logArchived = false;
>>> @@ -403,6 +434,7 @@
>>>          private CRC32 checksum = new CRC32(); // holder for the 
>>> checksum
>>>  
>>> +    protected static int CHECKSUM_SIZE = 8; // checksum size in bytes
>>>  
>>>          /**
>>> @@ -1194,8 +1226,28 @@
>>>              tf.resetTranId();
>>>          }
>>>  
>>> -        // done with recovery
>>> -
>>> +        // If we have reached here successfully, that means recovery 
>>> +        // was successful
>>> +        +        // Now that database has recovered successfully and 
>>> if it
>>> +        // is booted with derby.system.testMode=false mode, there is no
>>> +        // need to store information that database was booted 
>>> previously in +        // derby.system.testMode=true mode. +        
>>> // It is safe to update the marker in log.ctrl file to say
>>> +        // that the derby.system.testMode in not being used
>>> +   +        // Need to write to the log.ctrl file as well as the log 
>>> mirror file
>>> +    if( wasDBInTestModeNoSync && !logNotSynced )
>>> +        {
>>> +        // write to log control file and the mirror file that the 
>>> testmode +            // is not used now
>>> +            if(!updateTestModeFlagInCtrlFile((byte)0))
>>> +                throw StandardException.newException(
>>> +                    SQLState.LOG_CONTROL_FILE, getControlFileName());
>>> +           +            wasDBInTestModeNoSync = false;
>>> +        } +                 
>>> /////////////////////////////////////////////////////////////
>>>          // setup checktpoint daemon
>>>          /////////////////////////////////////////////////////////////
>>> @@ -1207,6 +1259,99 @@
>>>          }
>>>      }
>>>  
>>> +    /**
>>> +     * Update the flag that represents whether +     * 
>>> derby.system.testMode is enabled or not in both the log control file
>>> +     * as well as the mirror control file
>>> +     * <BR>
>>> +     * Basically, Read the control file, +     * update the flag to 
>>> the new passed in value +     * recalculate the checksum and write it 
>>> out to the log control file
>>> +     * <P> MT- synchronized by caller
>>> +     * @param value to set
>>> +     * @return false if the update didnt happen successfully, else true
>>> +     * @throws IOException
>>> +     * @throws StandardException
>>> +     * @see #readControlFile
>>> +     * @see #writeControlFile(StorageFile, long)
>>> +     */
>>> +    private boolean updateTestModeFlagInCtrlFile(byte value)
>>> +    {
>>> +        StorageRandomAccessFile logControlFile = null;
>>> +        byte barray[] = null;
>>> +        StorageFile ctrlFile = null;
>>> +        +        try
>>> +        {
>>> +            ctrlFile = getControlFileName();
>>> +            logControlFile = privRandomAccessFile(ctrlFile, "rw");
>>> +            if (!privCanWrite(ctrlFile))
>>> +                return false;
>>> +        +            // retrieve all except the checksum bytes
>>> +            barray = new byte[(int) logControlFile.length() - 
>>> CHECKSUM_SIZE];
>>> +            logControlFile.readFully(barray);
>>> +               +            if (SanityManager.DEBUG)
>>> +            {
>>> +                if (SanityManager.DEBUG_ON(DBG_FLAG))
>>> +                {
>>> +                    SanityManager.DEBUG(DBG_FLAG, 
>>> +                        "Updating derby.system.testMode marker 
>>> written out"+
>>> +                        "to log control file: prev value =  " +
>>> +                        barray[TESTMODE_NO_SYNC_FLAG_OFFSET] + " to 
>>> "+ value); +                }
>>> +            }
>>> +            // update the byte at position that represents whether 
>>> +            // derby.system.testMode was used
>>> +            barray[TESTMODE_NO_SYNC_FLAG_OFFSET] = value;
>>> +            +            // write back into file the updated value 
>>> and also compute +            // the newchecksum and write it to file
>>> +            logControlFile.seek(0);
>>> +            logControlFile.write(barray);
>>> +            checksum.reset();
>>> +            checksum.update(barray, 0, barray.length);
>>> +            logControlFile.writeLong(checksum.getValue());
>>> +            syncFile(logControlFile);
>>> +            logControlFile.close();
>>> +
>>> +            // write the same data to mirror control file
>>> +            ctrlFile = getMirrorControlFileName();
>>> +            logControlFile =
>>> +                    privRandomAccessFile(ctrlFile, "rw");
>>> +            if (!privCanWrite(ctrlFile))
>>> +                return false;
>>> +            logControlFile.seek(0);
>>> +            logControlFile.write(barray);
>>> +            logControlFile.writeLong(checksum.getValue());
>>> +            syncFile(logControlFile);
>>> +        }
>>> +        catch(IOException ioe)
>>> +        {
>>> +            logControlFile = null;
>>> +            return false;
>>> +        }
>>> +        catch(StandardException se)
>>> +        {
>>> +               logControlFile = null;
>>> +            return false;
>>> +        }
>>> +        finally
>>> +        {
>>> +           try
>>> +           { +               if (logControlFile != null)
>>> +                   logControlFile.close();
>>> +           }
>>> +           catch(IOException ioe2)
>>> +           {
>>> +               logControlFile = null;
>>> +               return false;
>>> +           }
>>> +        }
>>> +        return true;
>>> +    }
>>>      /**
>>>          Checkpoint the rawStore.
>>>  
>>> @@ -2142,10 +2287,61 @@
>>>  
>>>          //
>>>          // write some spare bytes after 2.0 we have 3 + 2(8) spare 
>>> bytes.
>>> +        // now with v10.1.0.0 we have 2 +2(8) spare bytes as one of 
>>> the spare
>>> +        // byte is used as a marker to write out if system is runinng
>>> +        // in derby.system.testMode=true or not.
>>> +                 long spare = 0;
>>> +        +        // A spare byte at offset TESTMODE_NO_SYNC_FLAG_OFFSET
>>> +        // is used to write out if database was in 
>>> derby.system.testMode=true mode
>>> +        // or not. +        // This byte will have one of the two 
>>> values
>>> +        // - TESTMODE_NO_SYNC_TRUE to represent database was booted
>>> +        //   in this mode +        // -  0 to represent database was 
>>> not booted in this mode
>>> +        //  +        // The sole purpose of this flag in control 
>>> file is to help
>>> +        // identify any recovery problems that might happen as a 
>>> result of this
>>> +        // mode being set. +        // +        // Possible 
>>> scenarios. In table below, cases are listed giving +        // the 
>>> value of derby.system.testMode if it was enabled or not at boot time
>>> +        // 
>>> ---------------------------------------------------------------
>>> +        //  Case   Previous   Current     Action , what should the 
>>> flag be set to in log.ctrl +        //          Boot       Boot       
>>> file
>>> +        // 
>>> --------------------------------------------------------------
>>> +        //   1       -         true        write 
>>> TESTMODE_NO_SYNC_TRUE +        //   2       -         false       
>>> write 0 +        //   3       false     false       write 0
>>> +        //   4       true      false       write 
>>> TESTMODE_NO_SYNC_TRUE +        //   5       true      true        
>>> write TESTMODE_NO_SYNC_TRUE
>>> +        // +        // Case 4 is the interesting case here.
>>> +        // one cannot write out the derby.system.testMode is 
>>> disabled just yet
>>> +        // because it is possible that the database recovery might 
>>> fail and on subsequent
>>> +        // restarts this state would be lost. So only when recovery 
>>> is successful, will +        // we update the flag in the log control 
>>> file to indicate that the derby.system.testMode
>>> +        // is not enabled.  (This will happen at the end of recovery 
>>> , see recover method) +        // +        // Case 1, 2 is when 
>>> database is created fresh with this property enabled.
>>> +        // +        // wasDBInTestModeNoSync has information if 
>>> database was previously booted
>>> +        // in derby.system.testMode=true mode
>>> +        if( !wasDBInTestModeNoSync ) // derby.system.testMode=false
>>> +        {
>>> +            // in this case, no need to write out any special marker
>>> +            // for derby.system.testMode=false case.
>>> +            if(!logNotSynced)
>>> +                daos.writeByte(0);  +            else
>>> +            daos.writeByte(TESTMODE_NO_SYNC_TRUE); +        }
>>> +        else
>>> +            daos.writeByte(TESTMODE_NO_SYNC_TRUE);
>>> +                 daos.writeByte(0);
>>>          daos.writeByte(0);
>>> -        daos.writeByte(0);
>>>          daos.writeLong(spare);
>>>          daos.flush();
>>>          // write the checksum for the control data written
>>> @@ -2281,7 +2477,24 @@
>>>                  onDiskMinorVersion = dais.readInt();
>>>                  int dbBuildNumber = dais.readInt();
>>>                  int flags = dais.readByte();
>>> -
>>> +               +        // check if the database was running 
>>> previously with
>>> +        // derby.system.testMode=true mode
>>> +                // if database was running previously in 
>>> derby.system.testMode=true
>>> +                // and if on a restart - recovery fails we want to 
>>> be able
>>> +                // to report that the boot error could have happened
>>> +                // because of this property
>>> +                wasDBInTestModeNoSync =(dais.readByte() & 
>>> TESTMODE_NO_SYNC_TRUE) != 0;
>>> +                +                if (SanityManager.DEBUG)
>>> +                {
>>> +                    if (SanityManager.DEBUG_ON(LogToFile.DBG_FLAG))
>>> +                        SanityManager.DEBUG(LogToFile.DBG_FLAG, 
>>> +                            "log control file, was 
>>> derby.system.testMode enabled = " + +                            
>>> wasDBInTestModeNoSync);
>>> +                }
>>> +                    +                                onDiskBeta = 
>>> (flags & IS_BETA_FLAG) != 0;
>>>  
>>>                  if (onDiskBeta)
>>> @@ -2794,19 +3007,15 @@
>>>              isWriteSynced = false;
>>>          }
>>>  
>>> -        if (Performance.MEASURE)
>>> +        // if system is running with derby.system.testMode enabled
>>> +                // log records and log file are not synced, so 
>>> +        // set flag to disable log syncs
>>> +        if (PropertyUtil.getSystemBoolean(Property.TESTMODE_NO_SYNC))
>>>          {
>>> -            // debug only flag - disable syncing of log.
>>> -            logNotSynced = -                
>>> PropertyUtil.getSystemBoolean(Property.STORAGE_LOG_NOT_SYNCED);
>>> -
>>> -            if (logNotSynced)
>>> -            {
>>> -                Monitor.logMessage("logNotSynced = true");
>>> -                //if log is Not being synced;files should not be 
>>> open in write sync mode
>>> -                isWriteSynced = false;
>>> -            }
>>> -           +            // disable syncing of log.
>>> +            logNotSynced = true;
>>> +              //if log is Not being synced;files should not be open 
>>> in write sync mode
>>> +            isWriteSynced = false;            }
>>>  
>>>          // try to access the log
>>> @@ -2830,6 +3039,18 @@
>>>                  if (privExists(logControlFileName))
>>>                  {
>>>                      checkpointInstant = 
>>> readControlFile(logControlFileName, startParams);
>>> +                    // in case system was running previously with 
>>> derby.system.testMode=true
>>> +                    // then print a message to the derby log
>>> +                    if(wasDBInTestModeNoSync)
>>> +            {
>>> +                // print message stating that the database was
>>> +            // previously running with derby.system.testMode=true
>>> +                // mode
>>> +                Monitor.logMessage(MessageService.getTextMessage(
>>> +                                MessageId.LOG_WAS_IN_TESTMODE_NO_SYNC,
>>> +                                Property.TESTMODE_NO_SYNC));
>>> +            }
>>> +                                            if (checkpointInstant == 
>>> LogCounter.INVALID_LOG_INSTANT &&
>>>                                          
>>> getMirrorControlFileName().exists())
>>>                          checkpointInstant =
>>> @@ -2879,8 +3100,9 @@
>>>                      // blow away the log file if possible
>>>                      if (!privDelete(logFile) && logFileNumber == 1)
>>>                      {
>>> +                        logErrMsgForTestModeNoSync();
>>>                          throw StandardException.newException(
>>> -                            SQLState.LOG_INCOMPATIBLE_FORMAT, 
>>> dataDirectory);
>>> +                            
>>> SQLState.LOG_INCOMPATIBLE_FORMAT,dataDirectory);
>>>                      }
>>>  
>>>                      // If logFileNumber > 1, we are not going to 
>>> write that file just
>>> @@ -2913,6 +3135,7 @@
>>>  
>>>                          if (!privDelete(logFile))
>>>                          {
>>> +                            logErrMsgForTestModeNoSync();
>>>                              throw StandardException.newException(
>>>                                      SQLState.LOG_INCOMPATIBLE_FORMAT,
>>>                                      dataDirectory);
>>> @@ -3584,15 +3807,8 @@
>>>              }
>>>              else
>>>              {
>>> -                if (Performance.MEASURE)
>>> -                {
>>> -                    if (!logNotSynced)
>>> -                        logOut.syncLogAccessFile();
>>> -                }
>>> -                else
>>> -                {
>>> -                    logOut.syncLogAccessFile();
>>> -                }
>>> +                if (!logNotSynced)
>>> +                    logOut.syncLogAccessFile();
>>>              }
>>>  
>>>              syncSuceed = true;
>>> @@ -3925,6 +4141,7 @@
>>>      */
>>>      protected void logErrMsg(String msg)
>>>      {
>>> +            logErrMsgForTestModeNoSync();
>>>          Monitor.logTextMessage(MessageId.LOG_BEGIN_ERROR);
>>>          Monitor.logMessage(msg);
>>>          Monitor.logTextMessage(MessageId.LOG_END_ERROR);
>>> @@ -3936,6 +4153,7 @@
>>>      */
>>>      protected void logErrMsg(Throwable t)
>>>      {
>>> +        logErrMsgForTestModeNoSync();
>>>          if (corrupt != null)
>>>          {
>>>              Monitor.logTextMessage(MessageId.LOG_BEGIN_CORRUPT_STACK);
>>> @@ -3951,6 +4169,22 @@
>>>          }
>>>      }
>>>  
>>> +    /**
>>> +     * In case of boot errors, and if database is either booted
>>> +     * with derby.system.testMode being set or was previously booted 
>>> in this
>>> +     * mode, mention in the error message that the error is probably 
>>> +     * because the derby.system.testMode was set. +     * Dont want 
>>> to waste time to resolve issues in such
>>> +     * cases
>>> +     * <p>
>>> +     * MT - not needed, informational only
>>> +     */
>>> +    protected void logErrMsgForTestModeNoSync()
>>> +    {
>>> +        if(logNotSynced || wasDBInTestModeNoSync)
>>> +            Monitor.logTextMessage(MessageId.LOG_TESTMODE_NO_SYNC_ERR);
>>> +    }
>>> +
>>>      private void printErrorStack(Throwable t)
>>>      {
>>>          Monitor.logMessage("-------------------------\n");
>>> Index: 
>>> java/engine/org/apache/derby/impl/store/raw/data/BaseDataFileFactory.java 
>>>
>>> ===================================================================
>>> --- 
>>> java/engine/org/apache/derby/impl/store/raw/data/BaseDataFileFactory.java    
>>> (revision 165265)
>>> +++ 
>>> java/engine/org/apache/derby/impl/store/raw/data/BaseDataFileFactory.java    
>>> (working copy)
>>> @@ -179,11 +179,11 @@
>>>      private static final String LINE =      
>>> "----------------------------------------------------------------";
>>>  
>>> -    // debug only flag - disable syncing of data during page 
>>> allocation.
>>> -    boolean dataNotSyncedAtAllocation;
>>> +    // disable syncing of data during page allocation.
>>> +    boolean dataNotSyncedAtAllocation = false;
>>>  
>>> -    // debug only flag - disable syncing of data during checkpoint.
>>> -    boolean dataNotSyncedAtCheckpoint;
>>> +    // disable syncing of data during checkpoint.
>>> +    boolean dataNotSyncedAtCheckpoint = false;
>>>  
>>>      // these fields can be accessed directly by subclasses if it 
>>> needs a
>>>      // different set of actions
>>> @@ -375,29 +375,24 @@
>>>  
>>>          droppedTableStubInfo = new Hashtable();
>>>  
>>> -        if (Performance.MEASURE)
>>> +        /**
>>> +         * If derby.system.testMode=true then set flags to +         
>>> * disable sync of data pages at allocation when file is grown,
>>> +         * disable sync of data writes during checkpoint
>>> +         */
>>> +        if (PropertyUtil.getSystemBoolean(Property.TESTMODE_NO_SYNC))
>>>          {
>>> -            // debug only flag - disable syncing of data during 
>>> checkpoint.
>>> -            dataNotSyncedAtCheckpoint = -                
>>> PropertyUtil.getSystemBoolean(
>>> -                    Property.STORAGE_DATA_NOT_SYNCED_AT_CHECKPOINT);
>>> +            // - disable syncing of data during checkpoint.
>>> +            dataNotSyncedAtCheckpoint = true;
>>>  
>>> -            // debug only flag - disable syncing of data during page 
>>> allocation.
>>> -            dataNotSyncedAtAllocation = -                
>>> PropertyUtil.getSystemBoolean(
>>> -                    Property.STORAGE_DATA_NOT_SYNCED_AT_ALLOCATION);
>>> -
>>> -            if (dataNotSyncedAtCheckpoint)
>>> -                Monitor.logMessage(
>>> -                    "Warning: " + -                    
>>> Property.STORAGE_DATA_NOT_SYNCED_AT_CHECKPOINT +
>>> -                    "set to true.");
>>> -
>>> -            if (dataNotSyncedAtAllocation)
>>> -                Monitor.logMessage(
>>> -                    "Warning: " + -                    
>>> Property.STORAGE_DATA_NOT_SYNCED_AT_ALLOCATION +
>>> -                    "set to true.");
>>> +            // - disable syncing of data during page allocation.
>>> +            dataNotSyncedAtAllocation = true;
>>> +            +            // log message stating that 
>>> derby.system.testMode
>>> +            // has been enabled.  +            
>>> Monitor.logMessage(MessageService.getTextMessage(
>>> +                MessageId.STORE_TESTMODE_NO_SYNC,
>>> +                Property.TESTMODE_NO_SYNC));
>>>          }
>>>  
>>>          fileHandler = new RFResource( this);
>>> Index: 
>>> java/engine/org/apache/derby/impl/store/raw/data/RAFContainer.java
>>> ===================================================================
>>> --- 
>>> java/engine/org/apache/derby/impl/store/raw/data/RAFContainer.java    
>>> (revision 165265)
>>> +++ 
>>> java/engine/org/apache/derby/impl/store/raw/data/RAFContainer.java    
>>> (working copy)
>>> @@ -325,15 +325,8 @@
>>>                  dataFactory.writeInProgress();
>>>                  try
>>>                  {
>>> -                    if (Performance.MEASURE)
>>> -                    {
>>> -                        if (!dataFactory.dataNotSyncedAtAllocation)
>>> -                            fileData.sync( false);
>>> -                    }
>>> -                    else
>>> -                    {
>>> +                    if (!dataFactory.dataNotSyncedAtAllocation)
>>>                          fileData.sync( false);
>>> -                    }
>>>                  }
>>>                  finally
>>>                  {
>>> @@ -504,16 +497,9 @@
>>>                      dataFactory.writeInProgress();
>>>                      inwrite = true;
>>>  
>>> -                    if (Performance.MEASURE)
>>> -                    {
>>> -                        if (!dataFactory.dataNotSyncedAtAllocation)
>>> -                            fileData.sync( false);
>>> -                    }
>>> -                    else
>>> -                    {
>>> +                    if (!dataFactory.dataNotSyncedAtAllocation)
>>>                          fileData.sync( false);
>>> -                    }
>>> -                }
>>> +                  }
>>>                  catch (IOException ioe)
>>>                  {
>>>                      // The disk may have run out of space. @@ 
>>> -629,15 +615,9 @@
>>>              dataFactory.writeInProgress();
>>>              try
>>>              {
>>> -                if (Performance.MEASURE)
>>> -                {
>>> -                    if (!dataFactory.dataNotSyncedAtCheckpoint)
>>> -                        file.sync( false);
>>> -                }
>>> -                else
>>> -                {
>>> -                    file.sync( false);
>>> -                }
>>> +                if (!dataFactory.dataNotSyncedAtCheckpoint)
>>> +                   file.sync( false);
>>> +
>>>              }
>>>              finally
>>>              {
>>> Index: java/engine/org/apache/derby/iapi/reference/MessageId.java
>>> ===================================================================
>>> --- java/engine/org/apache/derby/iapi/reference/MessageId.java    
>>> (revision 165265)
>>> +++ java/engine/org/apache/derby/iapi/reference/MessageId.java    
>>> (working copy)
>>> @@ -58,8 +58,10 @@
>>>      String LOG_CANNOT_CREATE_NEW_DUETO      = "L017";
>>>      String LOG_MAYBE_INCONSISTENT           = "L018";
>>>      String LOG_TRANS_LOG_SCAN               = "L019";
>>> +    String LOG_WAS_IN_TESTMODE_NO_SYNC      = "L020"; // system was 
>>> running in derby.system.testMode set +    String 
>>> LOG_TESTMODE_NO_SYNC_ERR         = "L021"; // hint that error could 
>>> be because +                                                      // 
>>> derby.system.testMode was set
>>>  
>>> -
>>>      /*
>>>       * Raw Store data
>>>       */
>>> @@ -75,6 +77,7 @@
>>>      String STORE_BACKUP_ABORTED             = "D010";
>>>      String STORE_REMOVED_BACKUP             = "D011";
>>>      String STORE_BACKUP_COMPLETED           = "D012";
>>> +    String STORE_TESTMODE_NO_SYNC           = "D013"; // for 
>>> derby.system.testMode is set
>>>  
>>>  
>>>      /*
>>> Index: java/engine/org/apache/derby/iapi/reference/Property.java
>>> ===================================================================
>>> --- java/engine/org/apache/derby/iapi/reference/Property.java    
>>> (revision 165265)
>>> +++ java/engine/org/apache/derby/iapi/reference/Property.java    
>>> (working copy)
>>> @@ -368,55 +368,47 @@
>>>          <P>
>>>      */
>>>      String STORAGE_TEMP_DIRECTORY = "derby.storage.tempDirectory";
>>> -
>>> +        /**
>>> -     * derby.storage.dataNotSyncedAtCheckPoint
>>> +     * derby.system.testMode
>>>       * <p>
>>> -     * When set, the store system will not force a sync() call on the
>>> -     * containers during a checkpoint.
>>> +     * When set, the store system will not force sync calls in the 
>>> following
>>> +     * cases  +     * - for the log file at each commit
>>> +     * - for the log file before data page is forced to disk
>>> +     * - for page allocation when file is grown
>>> +     * - for data writes during checkpoint
>>>       * <p>
>>> -     * An internal debug system only flag.  The recovery system will 
>>> not
>>> -     * work properly if this flag is enabled, it is provided to do 
>>> performance
>>> -     * debugging to see whether the system is I/O bound based on 
>>> checkpoint
>>> -     * synchronous I/O.
>>> +     * That means, when this property is set to true
>>> +     * - a commit no longer guarantees that the transaction's 
>>> modification
>>> +     *   will survive a system crash or JVM termination
>>> +     * - the database may not recover successfully upon restart
>>> +     * - a near full disk at runtime may cause unexpected errors
>>>       * <p>
>>> -     *
>>> -     **/
>>> -    public static final String STORAGE_DATA_NOT_SYNCED_AT_CHECKPOINT 
>>> = -        "derby.storage.dataNotSyncedAtCheckPoint";
>>> -
>>> -    /**
>>> -     * derby.storage.dataNotSyncedAtAllocation
>>> +     * This property is provided for performance reasons and should 
>>> ideally
>>> +     * only be used when there is no requirement for the database to 
>>> recover.
>>> +     * <BR> +     * One sample use would be to enable this property 
>>> when using Derby as a test
>>> +     * database, where recoverability is not a concern but 
>>> performance is important.
>>> +     * <BR>
>>> +     * Default value is false
>>> +     * <BR>
>>> +     * Valid Values
>>> +     * Valid values are true or false
>>> +     * <BR>
>>> +     * Example
>>> +     * derby.system.testMode=true
>>> +     * One can set this as a command line option to the JVM when 
>>> starting the
>>> +     * application or in the derby.properties file
>>> +     * <BR>
>>> +     * This property is static; if you change it while Derby is 
>>> running, +     * the change does not take effect until you reboot.  
>>>       * <p>
>>> -     * When set, the store system will not force a sync() call on the
>>> -     * containers when pages are allocated.
>>> -     * <p>
>>> -     * An internal debug system only flag.  The recovery system will 
>>> not
>>> -     * work properly if this flag is enabled, it is provided to do 
>>> performance
>>> -     * debugging to see whether the system is I/O bound based on 
>>> page allocation
>>> -     * synchronous I/O.
>>> -     * <p>
>>>       *
>>>       **/
>>> -    public static final String STORAGE_DATA_NOT_SYNCED_AT_ALLOCATION 
>>> = -        "derby.storage.dataNotSyncedAtAllocation";
>>> -
>>> -    /**
>>> -     * derby.storage.logNotSynced
>>> -     * <p>
>>> -     * When set, the store system will not force a sync() call on 
>>> the log at -     * commit.
>>> -     * <p>
>>> -     * An internal debug system only flag.  The recovery system will 
>>> not
>>> -     * work properly if this flag is enabled, it is provided to do 
>>> performance
>>> -     * debugging to see whether the system is I/O bound based on log 
>>> file
>>> -     * synchronous I/O.
>>> -     * <p>
>>> -     *
>>> -     **/
>>> -    public static final String STORAGE_LOG_NOT_SYNCED = -        
>>> "derby.storage.logNotSynced";
>>> -
>>> +    public static final String TESTMODE_NO_SYNC = +        
>>> "derby.system.testMode";
>>> +         /**
>>>       * derby.storage.fileSyncTransactionLog
>>>       * <p>
>>> Index: java/engine/org/apache/derby/loc/messages_en.properties
>>> ===================================================================
>>> --- java/engine/org/apache/derby/loc/messages_en.properties    
>>> (revision 165265)
>>> +++ java/engine/org/apache/derby/loc/messages_en.properties    
>>> (working copy)
>>> @@ -1150,7 +1150,10 @@
>>>  L016=cannot create a new log file {0}
>>>  L017=cannot create a new log file {0} due to {1}
>>>  L018=System may be in a inconsistent state, missing file {0}
>>> +L020=WARNING: The database was previously booted with {0} set to 
>>> true. Note that this mode {0}=true does not guarantee recoverability \n
>>> +L021=WARNING: The database was either previously booted or is 
>>> currently booted with {0} set to true. Note that this mode {0}=true 
>>> does not guarantee recoverability, and is probably what might have 
>>> caused this boot error \n
>>>  
>>> +
>>>  # Class Manager
>>>  C000=Wrote class {0} to file {1}. Please provide support with the 
>>> file and the following exception information: {2}
>>>  C001=Class {0} has a unknown certificate type in jar {1}, expecting 
>>> X509 certificate.
>>> @@ -1174,6 +1177,7 @@
>>>  D010=Backup aborted due to error
>>>  D011=removed old backup copy at {0}
>>>  D012=Backup completed, log instant at {0} \n
>>> +D013=WARNING: {0} is set to true. In this mode, there is no 
>>> guarantee that the system will recover a system crash or JVM 
>>> termination. Enable this property only when there is no need for 
>>> recoverability \n
>>>  
>>>  # Connectivity  J004=database identity
>>> Index: 
>>> java/testing/org/apache/derbyTesting/functionTests/tests/store/TestNoSyncMode.java 
>>>
>>> ===================================================================
>>> --- 
>>> java/testing/org/apache/derbyTesting/functionTests/tests/store/TestNoSyncMode.java    
>>> (revision 0)
>>> +++ 
>>> java/testing/org/apache/derbyTesting/functionTests/tests/store/TestNoSyncMode.java    
>>> (revision 0)
>>> @@ -0,0 +1,202 @@
>>> +/*
>>> + + Derby - Class 
>>> org.apache.derbyTesting.functionTests.tests.store.TestNoSyncs
>>> + + Copyright 2002, 2005 The Apache Software Foundation or its 
>>> licensors, as applicable.
>>> + + Licensed under the Apache License, Version 2.0 (the "License");
>>> + you may not use this file except in compliance with the License.
>>> + You may obtain a copy of the License at
>>> + + http://www.apache.org/licenses/LICENSE-2.0
>>> + + Unless required by applicable law or agreed to in writing, software
>>> + distributed under the License is distributed on an "AS IS" BASIS,
>>> + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 
>>> implied.
>>> + See the License for the specific language governing permissions and
>>> + limitations under the License.
>>> + + */
>>> +
>>> +package org.apache.derbyTesting.functionTests.tests.store;
>>> +
>>> +import java.sql.Connection;
>>> +import java.sql.Statement;
>>> +import java.sql.PreparedStatement;
>>> +import java.io.*;
>>> +/**
>>> + * This program tests the system when the derby.system.testMode 
>>> property is enabled
>>> + *
>>> + * When the derby.system.testMode is set, the system will not do any 
>>> sync to the disk and
>>> + * the recovery system will not work property.
>>> + * It is provided for performance reasons and should ideally only be 
>>> used
>>> + * when there is no requirement for the database to recover.
>>> + * <p>
>>> + * When set, the system will not do any syncs, the cases namely
>>> + * - no sync of the log file at each commit
>>> + * - no sync of the log file before data page is forced to disk
>>> + * - no sync of page allocation when file is grown
>>> + * - no sync of data writes during checkpoint
>>> + * <p>
>>> + * That means, when this property is set to true,
>>> + * - a commit no longer guarantees that the transaction's modification
>>> + *   will survive a system crash or JVM termination
>>> + * - the database may not recover successfully upon restart
>>> + * - a near full disk at runtime may cause unexpected errors
>>> + *
>>> + * This program tests for
>>> + * 1. enabling the derby.system.testMode is actually not doing the 
>>> syncs
>>> + *    by timing inserts
>>> + * 2. check if a warning message exists in derby.log
>>> + * 3. read log.ctrl file and check if the flag is set or not
>>> + *
>>> + *
>>> + * @author <a href="mailto:ksunithaghm@gmail.com">Sunitha 
>>> Kambhampati</a>
>>> + * @version 1.0
>>> + */
>>> +public class TestNoSyncMode
>>> +{
>>> +    public static void main(String[] args)
>>> +    {
>>> +        Connection conn = null;
>>> +        Statement s = null;
>>> +        PreparedStatement ps = null;
>>> +        try
>>> +        {
>>> +            // use the ij utility to read the property file and
>>> +            // make the initial connection.
>>> +            org.apache.derby.tools.ij.getPropertyArg(args);
>>> +            conn = org.apache.derby.tools.ij.startJBMS();
>>> +           +            s = conn.createStatement();
>>> +            s.execute("create table t1 (c1 int, c2 int)");
>>> +            s.close();
>>> +           +            // Test 1
>>> +            // this is a quick check incase someone breaks the 
>>> derby.system.testMode +            long timeTaken = 
>>> doInsertsInAutoCommit(conn);
>>> +            conn.close();
>>> +           +            int approxUpperBound = 3000; // approximate 
>>> upper bound in millisecond
>>> +            if ( timeTaken > approxUpperBound )
>>> +            {
>>> +                report("FAIL -- derby.system.testMode mode seems to 
>>> be broken.");
>>> +                report(" Time to insert rows in test exceeded the 
>>> usual limit.");
>>> +            }
>>> +           +            // Test 2
>>> +            // Check if derby.log has the warning message
>>> +            report("Is warning message about derby.system.testMode 
>>> present in derby.log ="+ isMessageInDerbyLog());
>>> +            // Test 3
>>> +            // Check if marker is correctly written out to 
>>> database   +            markerInControlFile();
>>> +           +           +        }
>>> +        catch (Throwable e) {
>>> +            report("FAIL -- unexpected exception: "+e);
>>> +            e.printStackTrace();
>>> +        }
>>> +       +    }
>>> +   +   +    /**
>>> +     * Note doing inserts in autocommit mode is probably the worst 
>>> case scenario
>>> +     * in terms of performance as each commit will involve a 
>>> flush/sync to disk
>>> +     * but in case of the derby.system.testMode=true mode, the syncs 
>>> dont happen
>>> +     * This test case times the inserts and assumes that the inserts 
>>> on any system
>>> +     * will be less than three second for 500 inserts.
>>> +         * Note this upper bound on time is just an approximate 
>>> estimation +     */
>>> +    public static long doInsertsInAutoCommit(Connection conn)
>>> +        throws Exception
>>> +    {
>>> +        PreparedStatement ps = conn.prepareStatement("insert into t1 
>>> values(?,?)");
>>> +        long count = 0;
>>> +       +        long start = System.currentTimeMillis();
>>> +       +        for ( int i = 0; i < 500; i++ )
>>> +        {
>>> +            ps.setInt(1,i);
>>> +            ps.setInt(2,i);
>>> +            count += ps.executeUpdate();
>>> +        }
>>> +       +        long end = System.currentTimeMillis();
>>> +        report("num successful inserts = "+count);
>>> +       +        return ( end - start);
>>> +    }
>>> +   +    /**
>>> +     * When derby.system.testMode is set, a warning message is 
>>> written out to derby.log
>>> +     * indicating that the property is set and that it does not 
>>> guarantee recoverability
>>> +     * This test tests if a message is written out to derby.log or 
>>> not +     */
>>> +    public static boolean isMessageInDerbyLog()
>>> +        throws Exception
>>> +    {
>>> +        BufferedReader reader = null;
>>> +        File derbylog = null;
>>> +        try
>>> +        {
>>> +            String derbyHome = System.getProperty("derby.system.home");
>>> +            derbylog = new File(derbyHome,"derby.log");
>>> +            reader = new BufferedReader(new FileReader(derbylog));
>>> +       +            String line = null;
>>> +            while((line = reader.readLine())!= null)
>>> +            {
>>> +                if(line.indexOf("derby.system.testMode") != -1)
>>> +                    return true;
>>> +           +            }
>>> +            return false;
>>> +        }
>>> +        finally
>>> +        {
>>> +            if (reader != null)
>>> +            {
>>> +                reader.close();
>>> +            }
>>> +            derbylog = null;
>>> +        }
>>> +    }
>>> +
>>> +    /**
>>> +     * if derby.system.testMode is set, a marker is written out into 
>>> log control file
>>> +     * to recognize that the database was previously booted in this 
>>> mode +      * Test if the marker byte is set correctly or not
>>> +      * See comments in org.apache.derby.impl.store.log.LogToFile 
>>> for IS_TESTMODE_NO_SYNC_FLAG
>>> +      */
>>> +    public static void  markerInControlFile()
>>> +        throws Exception
>>> +    {
>>> +        RandomAccessFile controlFile = null;
>>> +        try
>>> +        {
>>> +            int testModeNoSyncMarkerPosition = 29;
>>> +            String derbyHome = System.getProperty("derby.system.home");
>>> +            controlFile = new 
>>> RandomAccessFile(derbyHome+"/wombat/log/log.ctrl","r");
>>> +            controlFile.seek(testModeNoSyncMarkerPosition);
>>> +            report("log.ctrl file has testMode NoSync marker value = 
>>> "+ controlFile.readByte());
>>> +        }
>>> +        finally
>>> +        {
>>> +            if(controlFile != null)
>>> +                controlFile.close();
>>> +            +        }
>>> +    }
>>> +   +    public static void report(String msg)
>>> +    {
>>> +        System.out.println(msg);
>>> +    }
>>> +   +}
>>> +
>>> +
>>> +
>>>
>>> Property changes on: 
>>> java/testing/org/apache/derbyTesting/functionTests/tests/store/TestNoSyncMode.java 
>>>
>>> ___________________________________________________________________
>>> Name: svn:eol-style
>>>    + native
>>>
>>> Index: 
>>> java/testing/org/apache/derbyTesting/functionTests/tests/store/copyfiles.ant 
>>>
>>> ===================================================================
>>> --- 
>>> java/testing/org/apache/derbyTesting/functionTests/tests/store/copyfiles.ant    
>>> (revision 165265)
>>> +++ 
>>> java/testing/org/apache/derbyTesting/functionTests/tests/store/copyfiles.ant    
>>> (working copy)
>>> @@ -23,6 +23,7 @@
>>>  TransactionTable.sql
>>>  TransactionTable_app.properties
>>>  TransactionTable_derby.properties
>>> +TestNoSyncMode_derby.properties
>>>  access.sql
>>>  access_app.properties
>>>  access_derby.properties
>>> Index: 
>>> java/testing/org/apache/derbyTesting/functionTests/tests/store/TestNoSyncMode_derby.properties 
>>>
>>> ===================================================================
>>> --- 
>>> java/testing/org/apache/derbyTesting/functionTests/tests/store/TestNoSyncMode_derby.properties    
>>> (revision 0)
>>> +++ 
>>> java/testing/org/apache/derbyTesting/functionTests/tests/store/TestNoSyncMode_derby.properties    
>>> (revision 0)
>>> @@ -0,0 +1 @@
>>> +derby.system.testMode=true
>>>
>>> Property changes on: 
>>> java/testing/org/apache/derbyTesting/functionTests/tests/store/TestNoSyncMode_derby.properties 
>>>
>>> ___________________________________________________________________
>>> Name: svn:eol-style
>>>    + native
>>>
>>> Index: 
>>> java/testing/org/apache/derbyTesting/functionTests/master/TestNoSyncMode.out 
>>>
>>> ===================================================================
>>> --- 
>>> java/testing/org/apache/derbyTesting/functionTests/master/TestNoSyncMode.out    
>>> (revision 0)
>>> +++ 
>>> java/testing/org/apache/derbyTesting/functionTests/master/TestNoSyncMode.out    
>>> (revision 0)
>>> @@ -0,0 +1,3 @@
>>> +num successful inserts = 500
>>> +Is warning message about derby.system.testMode present in derby.log 
>>> =true
>>> +log.ctrl file has testMode NoSync marker value = 1
>>> Index: 
>>> java/testing/org/apache/derbyTesting/functionTests/suites/storemats.runall 
>>>
>>> ===================================================================
>>> --- 
>>> java/testing/org/apache/derbyTesting/functionTests/suites/storemats.runall    
>>> (revision 165265)
>>> +++ 
>>> java/testing/org/apache/derbyTesting/functionTests/suites/storemats.runall    
>>> (working copy)
>>> @@ -4,3 +4,4 @@
>>>  store/longColumn.sql
>>>  store/madhare.sql
>>>  store/streamingColumn.java
>>> +store/TestNoSyncMode.java
>>
>>
>>
>>
> 

Mime
View raw message