db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jack Klebanoff <kleba...@Mutagen.Net>
Subject [PATCH] Extension Packaging
Date Tue, 21 Sep 2004 22:05:44 GMT
I am an IBM employee working on extensions to Derby. I would like to
change Derby to simplify the packaging of extensions. An example of a
potential extension is RAM based storage.

There are several goals for the packaging mechanism.

   1. It should be simple for a user/DBA to set up. Ideally all that
      should be required is to plunk the jar file containing the
      extension into the Java class path.
   2. It should be possible to release extensions independently of base
      Derby releases, unless base Derby interfaces used by the extension
      change. The user should be able to use new Derby versions/fixes
      without updating the extension jar. This largely precludes
      combining the extension classes and resources with the base Derby
      classes and resources in one jar.
   3. The user should be able to select any number of extensions. This
      also precludes combining the extension classes and resources with
      the base Derby classes and resources in one jar.
   4. The footprint should be small.

The packaging solution proposed in this paper meets the first three
goals when there is no security manager. When a security manager is used
set up is not quite so simple: the security manager must configured to
permit the base Derby to read the extension jar files. I think that
this is unavoidable.

The packaging does a reasonable, though perhaps not optimal, job of
keeping the footprint small. The top level extension classes are loaded
before they are used and kept in the JVM even if the extension is not
used. It might be possible to remedy this with some work on the Monitor.

Under this packaging scheme an extension's compiled classes and a
properties file named "org/apache/derby/modules.properties" are packaged
in a jar file. The modules.properties file describes the extension to
the Derby monitor. When Derby starts up it reads all the
modules.properties files in its class path and uses the merged
properties lists as the description of the modules implementing the system.

The properties come in two flavors. The first is of the form:
    derby.module./extension-name/./id/=/module-class-name/
where /extension-name/ is the name of the extension, /id/ is a suffix
that makes the property name unique, and /module-class-name/ is the name
of a class that implements a module. The monitor will use the named
class as a candidate implementation when it searches for factory
implementations. See the javadoc for class
org.apache.derby.iapi.services.monitor.Monitor.

The second property flavor describes the implementation of a
sub-sub-protocol. It is of the form:
    derby.subSubProtocol./subSubProtocol-name/=/class-name/
where /subSubProtocol-name/ is the name of a sub-sub-protocol and
/class-name/ is the name of a class that implements either the
org.apache.derby.iapi.services.monitor.PersistentService interface or
the org.apache.derby.io.StorageFactory interface. For instance, suppose
that you implemented RAM storage with class com.mycom.ramStorageImpl.
You would provide a modules.properties file with the line
"derby.subSubProtocol.ram=com.mycom.ramStorageImpl". Then databases
opened with URLs starting with "jdbc:derby:ram:" would use
com.mycom.ramStorageImpl to implement their storage.

Currently the monitor reads one monitor.properties file which is
provided in the derby jar file. It understands the first flavor of
property. Currently sub-sub-protocol implementations are built in or
come from system properties.

This patch makes three changes.

   1. The monitor is changed to read multiple module.properties files,
      using method ClassLoader.getSystemResources.
   2. The monitor reads sub-sub-protocol properties from the
      modules.properties files, instead of just the system properties.
   3. Sub-sub-protocol property values can name either a StorageFactory
      or a PersistentService implementation. Previously it could only
      name a StorageFactory implementation.

svn diff:

Index: java/engine/org/apache/derby/impl/services/monitor/BaseMonitor.java
===================================================================
--- java/engine/org/apache/derby/impl/services/monitor/BaseMonitor.java	(revision 46949)
+++ java/engine/org/apache/derby/impl/services/monitor/BaseMonitor.java	(working copy)
@@ -43,6 +43,7 @@
  import org.apache.derby.iapi.services.io.FormatableInstanceGetter;
  import org.apache.derby.iapi.error.ExceptionSeverity;

+import  org.apache.derby.io.StorageFactory;

  import org.apache.derby.iapi.services.context.ErrorStringBuilder;

@@ -80,6 +81,8 @@
  import java.security.PrivilegedExceptionAction;
  import java.security.PrivilegedActionException;

+import java.net.URL;
+
  /**
  	Implementation of the monitor that uses the class loader
  	that the its was loaded in for all class loading.
@@ -1052,8 +1055,6 @@

  		Vector implementations = actualModuleList ? new Vector(moduleList.size()) : new Vector(0,1);

-		Class persistentServiceClass = PersistentService.class;
-
  		// Get my current JDK environment
  		int theJDKId = JVMInfo.JDK_ID;

@@ -1113,17 +1114,8 @@
  					Class possibleModule = Class.forName(className);

  					// Look for the monitors special modules, PersistentService ones.
-					if (persistentServiceClass.isAssignableFrom(possibleModule))  {
-						PersistentService ps = (PersistentService) newInstance(possibleModule);
-						if (ps == null) {
-							report("Class " + className + " cannot create instance, module ignored.");
-							continue;
-						}
-						if (serviceProviders == null)
-							serviceProviders = new Hashtable(3, (float) 1.0);
-						serviceProviders.put(ps.getType(), ps);
-						continue;
-					}
+					if (getPersistentServiceImplementation(possibleModule))
+                        continue;

  					// If this is a specific JDK version (environment) module
  					// then it must be ordered in the implementation list by envJDKId.
@@ -1199,8 +1191,35 @@
  					report("Class " + className + " " + le.toString() + ", module ignored.");
  				}
  			}
-		}
+            else if( key.startsWith( Property.SUB_SUB_PROTOCOL_PREFIX)) {
+                String subSubProtocol = key.substring( Property.SUB_SUB_PROTOCOL_PREFIX.length());
+                String className = moduleList.getProperty(key);

+				if (SanityManager.DEBUG && reportOn) {
+					report("Accessing module " + className + " to run initializers at boot time");
+				}
+                try {
+                    Class possibleImplementation = Class.forName(className);
+					// Look for the monitors special classes, PersistentService and StorageFactory ones.
+                    if( getPersistentServiceImplementation( possibleImplementation))
+                        continue;
+                    if( StorageFactory.class.isAssignableFrom( possibleImplementation)) {
+                        if( newInstance( possibleImplementation) == null)
+                            report("Class " + className + " cannot create instance, StorageFactory
ignored.");
+                        else
+                            storageFactories.put( subSubProtocol, className);
+                        continue;
+                    }
+                }
+				catch (ClassNotFoundException cnfe) {
+					report("Class " + className + " " + cnfe.toString() + ", module ignored.");
+				}
+				catch (LinkageError le) {
+					report("Class " + className + " " + le.toString() + ", module ignored.");
+				}
+            }
+        }
+
  		if (implementations.isEmpty())
  			return null;
  		implementations.trimToSize();
@@ -1208,40 +1227,79 @@
  		return implementations;
  	}

+    private boolean getPersistentServiceImplementation( Class possibleModule)
+    {
+        if( ! PersistentService.class.isAssignableFrom(possibleModule))
+            return false;
+
+        PersistentService ps = (PersistentService) newInstance(possibleModule);
+        if (ps == null) {
+            report("Class " + possibleModule.getName() + " cannot create instance, module
ignored.");
+        } else {
+            if (serviceProviders == null)
+                serviceProviders = new Hashtable(3, (float) 1.0);
+            serviceProviders.put(ps.getType(), ps);
+        }
+        return true;
+    } // end of getPersistentServiceImplementation
+
  	protected Vector getDefaultImplementations() {

-		InputStream is = loadModuleDefinitions();
+		Properties moduleList = new Properties();
+        boolean firstList = true;

-		if (is == null) {
+        try {
+            for( Enumeration e = ClassLoader.getSystemResources( "org/apache/derby/modules.properties");
+                 e.hasMoreElements() ;) {
+                URL modulesPropertiesURL = (URL) e.nextElement();
+                InputStream is = null;
+                try {
+                    is = loadModuleDefinitions( modulesPropertiesURL);
+                    if( firstList) {
+                        moduleList.load( is);
+                        firstList = false;
+                    }
+                    else {
+                        // Check for duplicates
+                        Properties otherList = new Properties();
+                        otherList.load( is);
+                        for( Enumeration newKeys = otherList.keys(); newKeys.hasMoreElements()
;)
+                        {
+                            String key = (String) newKeys.nextElement();
+                            if( moduleList.contains( key))
+                                // RESOLVE how do we localize messages before we have finished
initialization?
+                                report( "Ignored duplicate property " + key + " in " + modulesPropertiesURL.toString());
+                            else
+                                moduleList.setProperty( key, otherList.getProperty( key));
+                        }
+                    }
+                } catch (IOException ioe) {
+                    if (SanityManager.DEBUG)
+                        report("Can't load implementation list " + modulesPropertiesURL.toString()
+ ": " + ioe.toString());
+                } finally {
+                    try {
+                        if( is != null)
+                            is.close();
+                    } catch (IOException ioe2) {
+                    }
+                }
+            }
+        } catch (IOException ioe) {
+            if (SanityManager.DEBUG)
+                report("Can't load implementation list: " + ioe.toString());
+        }
+        if( firstList) {
  			if (SanityManager.DEBUG)
  				report("Default implementation list not found");
  			return null;
  		}

-		Properties moduleList = new Properties();
-
-		try {
-
-			moduleList.load(is);
-
-		} catch (IOException ioe) {
-			if (SanityManager.DEBUG)
-				report("Can't load default implementation list: " + ioe.toString());
-			return null;
-		} finally {
-
-			try {
-				is.close();
-			} catch (IOException ioe2) {
-			}
-		}
-
  		return getImplementations(moduleList, true);
-	}
+	} // end of getDefaultImplementations

-	protected InputStream loadModuleDefinitions() {
+	protected InputStream loadModuleDefinitions( URL propertyFileURL) throws IOException {
  		// SECURITY PERMISSION - IP1
-		return getClass().getResourceAsStream("/org/apache/derby/modules.properties");
+		return propertyFileURL.openStream();
  	}

  	/*
@@ -1595,10 +1653,10 @@
              className = PropertyUtil.getSystemProperty( propertyName);
          if( className != null)
              return className;
-        return (String) builtInStorageFactory.get( subSubProtocol);
+        return (String) storageFactories.get( subSubProtocol);
      } // end of getStorageFactoryClassName

-    private static final HashMap builtInStorageFactory = new HashMap();
+    private static final HashMap storageFactories = new HashMap();
      static {
  		String dirStorageFactoryClass;
  		if( JVMInfo.JDK_ID >= 4)
@@ -1607,14 +1665,14 @@
              dirStorageFactoryClass = "org.apache.derby.impl.io.DirStorageFactory";


-        builtInStorageFactory.put( PersistentService.DIRECTORY, dirStorageFactoryClass);
-        builtInStorageFactory.put( PersistentService.CLASSPATH,
+        storageFactories.put( PersistentService.DIRECTORY, dirStorageFactoryClass);
+        storageFactories.put( PersistentService.CLASSPATH,
                                  "org.apache.derby.impl.io.CPStorageFactory");
-        builtInStorageFactory.put( PersistentService.JAR,
+        storageFactories.put( PersistentService.JAR,
                                  "org.apache.derby.impl.io.JarStorageFactory");
-        builtInStorageFactory.put( PersistentService.HTTP,
+        storageFactories.put( PersistentService.HTTP,
                                  "org.apache.derby.impl.io.URLStorageFactory");
-        builtInStorageFactory.put( PersistentService.HTTPS,
+        storageFactories.put( PersistentService.HTTPS,
                                  "org.apache.derby.impl.io.URLStorageFactory");
      }

Index: java/engine/org/apache/derby/impl/services/monitor/FileMonitor.java
===================================================================
--- java/engine/org/apache/derby/impl/services/monitor/FileMonitor.java	(revision 46949)
+++ java/engine/org/apache/derby/impl/services/monitor/FileMonitor.java	(working copy)
@@ -23,6 +23,8 @@
  import java.io.IOException;
  import java.io.InputStream;

+import java.net.URL;
+
  /**
  	Implementation of the monitor that uses the class loader
  	that the its was loaded in for all class loading.
@@ -154,6 +156,7 @@
  	private String key3;
  	private Runnable task;
  	private int intValue;
+    private URL propertyFileURL;

  	/**
  		Initialize the system in a privileged block.
@@ -170,13 +173,19 @@
  		}
  	}

-	protected synchronized final InputStream loadModuleDefinitions() {
+	protected synchronized final InputStream loadModuleDefinitions( URL propertyFileURL) throws
IOException {
  		action = 2;
+        this.propertyFileURL = propertyFileURL;
  		try {
  			return (InputStream) java.security.AccessController.doPrivileged(this);
          } catch (java.security.PrivilegedActionException pae) {
-			throw (RuntimeException) pae.getException();
-		}
+            Exception e = pae.getException();
+            if( e instanceof IOException)
+                throw (IOException) e;
+			throw (RuntimeException) e;
+		} finally {
+            this.propertyFileURL = null;
+        }
  	}

  	public synchronized final String getJVMProperty(String key) {
@@ -247,7 +256,7 @@
  			return new Boolean(PBinitialize(action == 0));
  		case 2:
  			// SECURITY PERMISSION - IP1
-			return super.loadModuleDefinitions();
+			return super.loadModuleDefinitions( propertyFileURL);
  		case 3:
  			// SECURITY PERMISSION - OP1
  			return PBgetJVMProperty(key3);


Mime
View raw message