Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: <derby-dev@db.apache.org>
Received-SPF: pass (asf.osuosl.org: local policy)
Date: Fri, 19 May 2006 23:49:35 +0200
From: Olav Sandstaa <Olav.Sandstaa@Sun.COM>
Subject: Re: jdk16 and suite derbyall - 128 failures
In-reply-to: <446C76EC.4060500@sun.com>
To: derby-dev@db.apache.org
Message-id: <446E3D6F.5030605@sun.com>
MIME-version: 1.0
Content-type: text/plain; charset=ISO-8859-1; format=flowed
Content-transfer-encoding: 7BIT
User-Agent: Thunderbird 1.5.0.2 (X11/20060420)
References: <836ae9af0605101709w12a02835n7aabf25b95e5c8bd@mail.gmail.com>
 <446297B2.7080009@sun.com> <446C27DF.4030603@sun.com>
 <446C62DE.7020207@sun.com> <446C76EC.4060500@sun.com>

I have not found the real cause for why the Nist tests fail but here is a
short re-cap of my current findings - in case anyone else is looking at this
problem or know this code better than me and have suggestions on what is 
wrong. 

1. The security exception that makes the test fail (see earlier email) is
   happening because the variable home in DirStorageFactory.doInit() which
   should contain the path for the directory where the test should run 
in null
   (see also earlier email).

2. The "home" variable should be initialized to the correct value in
   the constructor of the StorageFactoryService class. This contains the
   following piece of code:

       Object monitorEnv = Monitor.getMonitor().getEnvironment();
       if (monitorEnv instanceof File) {
           ....

   Unfortunately the first of these two lines returns a null value (instead
   of a File object as it should) making the "home" variable not getting
   initialized. (Comment: I think this code should be extended with
   an "else" part to this if statement making Derby be more "fail fast" in
   this situation? ).

   So why do we not get a File object here? The answer to this question is
   that I do not think the FileMonitor is created. And why is the 
FileMonitor
   not created?

3. If I understand the code correctly, the FileMonitor is created during
   loading/booting of the JDBC driver. The EmbeddedDriver.java file 
contains to methods that boots the driver:

   a. Static code:

       public class EmbeddedDriver implements Driver {

    static {
            System.out.println("EmbeddedDriver.static"); // by Olav
        EmbeddedDriver.boot();
    }

   b. The constructor:

        // Boot from the constructor as well to ensure that
    // Class.forName(...).newInstance() reboots Derby
    // after a shutdown inside the same JVM.
    public EmbeddedDriver() {
            System.out.println("EmbeddedDriver.EmbeddedDriver"); // by Olav
            EmbeddedDriver.boot();
    }

   Both these "boots" the JDBC driver and eventually calls the method
   JDBCBoot.boot. This method contains the following code that should
   get the monitors loaded:

    public void boot(String protocol, PrintStream logging) {

       if (org.apache.derby.jdbc.InternalDriver.activeDriver() == null)
        {
            // request that the InternalDriver (JDBC) service and the
            // authentication service be started.
            //
            addProperty("derby.service.jdbc", 
"org.apache.derby.jdbc.InternalDriver");
            addProperty("derby.service.authentication", 
AuthenticationService.MODULE);

            Monitor.startMonitor(bootProperties, logging);


   When the Nist tests fails, the part of this code that do
   Monitor.startMonitor is not run (at least not as part of the Nist test
   itself).

That was probably more than I planned to say about the code. Then some
observations:

A. When the Nist test succeed (by backing out DERBY-930 or removing the DB2
   driver or ....) I see that:

     a. The static method of the EmbeddedDriver is executed

     b. org.apache.derby.jdbc.InternalDriver.activeDriver() returns null.

     c. and the JDBCBoot.boot() methods starts the Monitors.

     d. The EmbeddedDriver's constructor is run.

B. When the Nist test fails I see:

     a. The static method of the EmbeddedDriver is NOT executed (at least
        not as part of the Nist test, but it could have been run earlier
        by the framework?)

     b. The EmbeddedDriver's constructor is run.

     c. org.apache.derby.jdbc.InternalDriver.activeDriver() returns
        NOT null (ie. it already have a driver)

     d. The JDBCBoot.boot() does not starts the Monitors (due to c.)

   It seems like we "some time earlier" in the VM's life (possible during
   startup or run of one of the other suites?) that:

    1. The static methdo of the EmbeddedDriver has been executed (or 
MAYBE IT
       IS NEVER executed?)

    2. A driver has been loaded since InternalDriver.activeDriver() returns
       a driver (from where?)

    3. The FileMonitor has not been started. Why not?

A last observation is:

C. The Nist suite is the only suite that has useprocess=false defined. If
   I remove this from the property file it seems like the tests run ok.

Any feedback on this, suggestions and questions are highly welcome - I am on
thin ice on this :-)

Regards,
Olav


Olav Sandstaa wrote:
> The AccessControlException thrown in DirStorageFactory.doInit() is caused
> by the "home" (directory) variable having the value null.
>
> As pointed out by Ole, the Nist tests started to fail in revision
> r396638 (see DERBY-930). I have run derbyall for r396637 and r396638
> with and without the DB2 driver in the path and printed the values of
> the home and dataDirectory variables used by
> DirStorageFactory.doInit():
>
>
>         without DB2 driver                        with DB2 driver
> ---------------------------------------------------------------------------- 
>
> r396637: home: /export/home/tmp/derbysuite/nist    home: 
> /export/home/tmp/derbysuite/nist
>         dataDirectory: wombat                     dataDirectory: wombat
>
> r396638: home: /export/home/tmp/derbysuite/nist    home: null
>         dataDirectory: wombat                     dataDirectory: wombat
>
>
> So basically the Nist tests fail due to not knowing which directory to
> create the database in.
>
> ..olav
>
> Olav Sandstaa wrote:
>> The exception causing the Nist tests to fail when running derbyall 
>> using jdk1.6 has the following call stack:
>>
>> java.security.AccessControlException: access denied 
>> (java.util.PropertyPermission user.dir read)
>>    at 
>> java.security.AccessControlContext.checkPermission(AccessControlContext.java:323) 
>>
>>    at 
>> java.security.AccessController.checkPermission(AccessController.java:546) 
>>
>>    at 
>> java.lang.SecurityManager.checkPermission(SecurityManager.java:532)
>>    at 
>> java.lang.SecurityManager.checkPropertyAccess(SecurityManager.java:1285)
>>    at java.lang.System.getProperty(System.java:652)
>>    at java.io.UnixFileSystem.resolve(UnixFileSystem.java:118)
>>    at java.io.File.getCanonicalPath(File.java:559)
>>    at 
>> org.apache.derby.impl.io.DirStorageFactory.doInit(DirStorageFactory.java:190) 
>>
>>    at 
>> org.apache.derby.impl.io.BaseStorageFactory.init(BaseStorageFactory.java:87) 
>>
>>    at 
>> org.apache.derby.impl.services.monitor.StorageFactoryService.privGetStorageFactoryInstance(StorageFactoryService.java:201) 
>>
>>    at 
>> org.apache.derby.impl.services.monitor.StorageFactoryService.access$400(StorageFactoryService.java:64) 
>>
>>    at 
>> org.apache.derby.impl.services.monitor.StorageFactoryService$11.run(StorageFactoryService.java:774) 
>>
>>    at java.security.AccessController.doPrivileged(Native Method)
>>    at 
>> org.apache.derby.impl.services.monitor.StorageFactoryService.getCanonicalServiceName(StorageFactoryService.java:768) 
>>
>>    at 
>> org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(BaseMonitor.java:1537) 
>>
>>    at 
>> org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(BaseMonitor.java:990) 
>>
>>    at 
>> org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Monitor.java:541) 
>>
>>    at 
>> org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(EmbedConnection.java:1591) 
>>
>>    at 
>> org.apache.derby.impl.jdbc.EmbedConnection.<init>(EmbedConnection.java:216) 
>>
>>    at 
>> org.apache.derby.impl.jdbc.EmbedConnection30.<init>(EmbedConnection30.java:72) 
>>
>>    at 
>> org.apache.derby.impl.jdbc.EmbedConnection40.<init>(EmbedConnection40.java:47) 
>>
>>    at 
>> org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Driver40.java:64)
>>    at 
>> org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:200)
>>    at java.sql.DriverManager.getConnection(DriverManager.java:548)
>>    at java.sql.DriverManager.getConnection(DriverManager.java:148)
>>    at org.apache.derby.impl.tools.ij.util.startJBMS(util.java:516)
>>    at org.apache.derby.impl.tools.ij.util.startJBMS(util.java:616)
>>    at 
>> org.apache.derby.impl.tools.ij.ConnectionEnv.init(ConnectionEnv.java:64)
>>    at org.apache.derby.impl.tools.ij.utilMain.<init>(utilMain.java:176)
>>    at 
>> org.apache.derby.impl.tools.ij.utilMain14.<init>(utilMain14.java:51)
>>    at org.apache.derby.impl.tools.ij.Main14.getutilMain(Main14.java:102)
>>    at org.apache.derby.impl.tools.ij.Main.<init>(Main.java:265)
>>    at org.apache.derby.impl.tools.ij.Main14.<init>(Main14.java:68)
>>    at org.apache.derby.impl.tools.ij.Main14.getMain(Main14.java:91)
>>    at org.apache.derby.impl.tools.ij.Main.mainCore(Main.java:189)
>>    at org.apache.derby.impl.tools.ij.Main14.main(Main14.java:55)
>>    at org.apache.derby.tools.ij.main(ij.java:60)
>>    at org.apache.derbyTesting.functionTests.harness.RunIJ.run(Unknown 
>> Source)
>>    at java.lang.Thread.run(Thread.java:619)
>>
>> I am working on to figure out why this happens when running with 
>> jdk1.6 opposed to when running with earlier jdk versions. Since I am 
>> not very familiar with the security manager configuration used by the 
>> tests I appreciate all suggestions that other might have on why this 
>> happens.
>>
>> Regards,
>> Olav
>