From derby-dev-return-232-apmail-db-derby-dev-archive=db.apache.org@db.apache.org Sat Sep 04 01:09:01 2004 Return-Path: Delivered-To: apmail-db-derby-dev-archive@www.apache.org Received: (qmail 82634 invoked from network); 4 Sep 2004 01:09:01 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur-2.apache.org with SMTP; 4 Sep 2004 01:09:01 -0000 Received: (qmail 80027 invoked by uid 500); 4 Sep 2004 01:09:00 -0000 Delivered-To: apmail-db-derby-dev-archive@db.apache.org Received: (qmail 80003 invoked by uid 500); 4 Sep 2004 01:09:00 -0000 Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm Precedence: bulk list-help: list-unsubscribe: list-post: List-Id: "Derby Development" Reply-To: "Derby Development" Delivered-To: mailing list derby-dev@db.apache.org Received: (qmail 79991 invoked by uid 99); 4 Sep 2004 01:09:00 -0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received-SPF: pass (hermes.apache.org: local policy) Received: from [204.146.167.214] (HELO Boron.MeepZor.Com) (204.146.167.214) by apache.org (qpsmtpd/0.28) with ESMTP; Fri, 03 Sep 2004 18:08:58 -0700 Received: from [9.30.40.215] (dmz-firewall [206.199.198.4]) by Boron.MeepZor.Com (8.11.6/8.11.6) with ESMTP id i8418wG24317 for ; Fri, 3 Sep 2004 21:08:58 -0400 Message-ID: <413914FD.1020802@Source-Zone.org> Date: Fri, 03 Sep 2004 18:06:05 -0700 From: Suresh Thalamati Reply-To: tsuresh@Source-Zone.org User-Agent: Mozilla Thunderbird 0.7.3 (Windows/20040803) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Derby Development Subject: Re: java1.4.2 "rws" mode fix: LogToFile.java diff References: <6A8D4356-FBDD-11D8-A91C-000D93ADDBD4@serv.net> <509F159B-FC87-11D8-AF75-000D93ADDBD4@serv.net> <41375019.7000404@sbcglobal.net> In-Reply-To: <41375019.7000404@sbcglobal.net> X-Enigmail-Version: 0.85.0.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N Tested this case with a simple Java program that opens a file in "rw" mode first and then reopens in "rws": mode similar to the calls derby logging system makes. It also failed with the similar error on latest jdk142 (build 1.4.2_05-1) from Apple: Exception in thread "main" Java.io.FileNotFoundException: log.dat (File exists) Test program works fine on Win2k with jdk142. I think this is bug in latest jdk142 from Apple. Work around for this problem is disabling the write sync mechanism for transaction log. Write sync mode can be disables by setting "derby.storage.fileSyncTransactionLog=true" in derby.properties. When this property is set to true logging system does file syncs on commits similar to what happens on jvms before jdk142. Insert/deletes/updated ..etc can also fail on databases that were created with earlier jvm's when booted with the latest jvm because log switch also make similar I/O calls. If write sync mode is disabled on these database, they will work fine. Write Sync Test Java Program: import java.io.*; /** * Test synchronous writes on MAC with JDK1.4.2(updated 1). */ public class TestWriteSync { public static void main(String [] args) throws Exception { RandomAccessFile raf; File file = new File("log.dat"); if (file.exists()) file.delete(); byte[] data = new byte[4096]; ; // open the file in rw mode raf = new RandomAccessFile(file,"rw"); //just write some bytes raf.write(data); raf.close(); // reopen the above file in write sync mode // Note: following call is failing on MAC raf = new RandomAccessFile(file,"rws"); raf.seek(4096); //just write some bytes raf.write(data); raf.close(); System.out.println("RWS MODE IS WORKING FINE"); } } -suresh Mike Matrigali wrote: > Joe I don't have access to an OSX 10.3.5. This error is strange, from > my reading of the java interfaces and the code the first call is > creating the file, and the second call should just be opening the > existing file - and should not be getting a file exists error. > > I have tested this on a sun jdk 1.4.2 and an ibm 1.42 on windows and I > believe it has also been tested against a linux jdk 1.4.2. > > Do you have any time to write a test case of the I/O calls separate > from derby? This has the feel of a JVM bug, but maybe the interfaces > are not being used correctly. This is a relatively new feature in > jvm's (and new code in derby) so could be a bug in either area. > > > It would be nice if a work around can be found to get this up and > running in the OSX environment but not have to pay the extra > performance cost across all the other JVM's. > > Joseph Grace wrote: > >> Dear Suresh: >> >> Thank you for your informative reply and explanations of the original >> code. Much appreciated. I am relearning java and learning derby as >> I go here, so I'll answer your questions as I can. Add salt to taste >> as most of my "knowledge" I gleaned from the comments in the code I >> modified. >> >> FYI, I am using the most up-to-date OSX 10.3.5 and ditto for OSX java: >> >> java version "1.4.2_05" >> Java(TM) 2 Runtime Environment, Standard Edition 4(build 1.4.2_05-11) >> Java HotSpot(TM) Client VM (build 1.4.2-38, mixed mode) >> >> and I have not tested on any other platform. >> >> >> I believe your assessments below are correct. >> >> Suresh Thalamati wrote: >> >>> I am trying to understand the problem you described above. What is >>> the error you are seeing when you try to create >>> the database in the Mac environment. Are u seeing this problem >>> with JDK1.4.2 on any other OS ? >>> Could you please explain more on how database can be created >>> twice ? >>> From the changes described here, what I find is you are >>> trying to >>> avoid open/create >>> of log file in "rw" mode first and then in "rws" mode by >>> making >>> it a single privRandomAccess File call: >>> >>> >> StorageRandomAccessFile theLog = privRandomAccessFile( logFile, >>> isWriteSynced? "rws": "rw"); >> >> >> >> Yes, I merged the file creations to get the code working (and avoid a >> "File exists" exception: >> >> ~/derby$ java -cp jars/sane/derby.jar:jars/sane/derbytools.jar >> org.apache.derby.tools.ij >> ij version 10.0 (C) Copyright IBM Corp. 1997, 2004. >> ij> connect 'jdbc:derby:test;create=true'; >> ERROR XJ041: Failed to create database 'test', see the next exception >> for details. >> ERROR XBM01: Startup failed due to an exception, see next exception >> for details. >> ERROR XJ001: Java exception: >> '/Users/occam/dev/java/derby/test/log/log1.dat (File exists): >> java.io.FileNotFoundException'. >> ij> >> >> I thought there may have been some rush changes in the handoff from >> IBM to Apache which left a bug overlooked in java 1.4.2 (mea culpa). >> I did not realize the double-open was intended as an optimization >> (since it fails on my system and so seemed a bug). >> >> The error is a little misleading but, looking under the covers, I was >> able to confirm the existence of "log1.dat" (or somesuch) at the time >> of the error report attempt to recreate/open "log1.dat". So, "File >> exists" is the correct complaint. I have no idea whether that's >> in/correct behavior. If there is incorrect behavior on OSX's java >> 1.4.2_05, please describe it, and I shall submit it to Apple (or you >> can, as desired). >> >>> I think the following part of the changes might increase database >>> creation time when write sync is enabled.. >> >> >> >> Aha. Yes, it does take quite a long time now that you mention it. >> I'm glad that may not be normal. >> >>> Please correct me if my observation is not right . I think the >>> Reason behind opening files in "RW" mode first >>> and then reopen in "RWS" might have been to make preallocation of >>> the log file finish faster. >>> . Preallocation of the log file by doing writes to a file opened in >>> "rws" mode will be much slower than >>> doing writes to file opened in "rw" mode . >> >> >> >> Sounds good to me (i.e., I'll take your word for it :-). >> >> Apologies for any confusion. Please let me know whether this fixes a >> bug or not, or whether I should submit a bug report to Apple for OSX >> java instead? >> >> Thanks, >> >> = Joe = >> >> >