db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cheng Che Chen (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (DERBY-646) In-memory backend storage support
Date Sun, 22 Feb 2009 23:11:01 GMT

    [ https://issues.apache.org/jira/browse/DERBY-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12675713#action_12675713
] 

chatom edited comment on DERBY-646 at 2/22/09 3:09 PM:
---------------------------------------------------------------

Hi folks,

Attached (derby-646-20090222.stat, derby-646-20090222.diff) is my implementation of the interfaces
org.apache.derby.io.{StorageFile,StorageRandomAccessFile,WritableStorageFactory} such that
all file system operations are emulated entirely in memory, producing no file or directory
on disk.

A brief description of the content:

- java\engine\org\apache\derby\impl\io\memory

  This directory contains the source files for the implementation of the database storage
factory interfaces.

- java\testing\org\apache\derbyTesting\functionTests\tests\memoryIO

  This directory contains the source files for the JUnit tests.  It consists of three sets
of fairly extensive unit tests of the basic building blocks of my implementation.  They started
out as five sets of stand-alone JUnit tests and were converted to conform to Derby's JUnit
test framework.  In the process of conversion, two sets of stand-alone tests were dropped
because they use on-disk operations to verify the correctness of in-memory operations, and
Derby's test framework does not allow arbitrary operations on disk, as far as I can tell.
 These unit tests run only in the embedded mode of Derby's test framework, because they make
no DB connections.

  In addition, there is one functional test consisting of a simple but non-trivial clustering
application.  It is non-trivial because clustering is an important application in many engineering
and scientific disciplines; it is simple in that it computes clustering of numerical data
in one dimensional space -- the simplest case.  The application implements the same algorithm
in both JAVA (computed by the JUnit test case object itself) and SQL (computed by Derby using
the in-memory storage factory) and cross-checks the computed results, within some floating-point
tolerance.  The SQL portion of the application computes quite a few CREATE, INSERT, DELETE,
SELECT, JOIN, GROUP BY, ORDER BY, DROP; and so should exercise the storage layer to a good
extent.  This functional test runs in both the embedded mode and the client mode of Derby's
test framework.

The sub-sub-protocol associated with this in-memory storage factory is named "memory".  So
the connection string looks like "jdbc:derby:memory:MyTestDb" or "jdbc:derby://localhost:1527/memory:MyTestDb".

-----

HELP!!!  I've gotten the tests compiled properly with "ant all".  The compiled class files
show up under "classes\org\apache\derbyTesting\functionTests\tests\memoryIO".  But again I'm
running into the problem of not knowing how to get them included in the "derbyTesting.jar"
file.

Nor did I figure out how to have them automatically executed with "ant junit-all", "ant junitreport",
etc.  For now, this is what I do after "ant all" to execute the tests on Windows:

  java -cp .\tools\java\junit.jar;.\classes junit.textui.TestRunner org.apache.derbyTesting.functionTests.tests.memoryIO._Suite

(BTW, when I execute the above command, the disk activity LED on my computer goes wild during
the functional test.  That doesn't happen with the stand-alone version of the same functional
test (before conversion to run under Derby's test framework).  Perhaps I have mis-configured
something in the test object itself such that the above command does not actually test the
in-memory storage factory?)

-----

Existing files modified by this patch:

- java\engine\org\apache\derby\impl\build.xml
  tools\jar\extraDBMSclasses.properties

  These two files were modified so that my implementation would be compiled and included in
Derby's JAR files.

- java\engine\org\apache\derby\iapi\services\monitor\PersistentService.java
  java\engine\org\apache\derby\impl\services\monitor\BaseMonitor.java
  
  These two files were modified only because I did not feel like typing "-Dderby.subSubProtocol.memory=org.apache.derby.impl.io.memory.MemoryStorageFactory"
for every run.  I simply added a mapping from the "memory" service to the "org.apache.derby.impl.io.memory.MemoryStorageFactory"
class in BaseMonitor.

- java\testing\build.xml

  This file was modified so that my tests would be compiled into class files under "classes\org\apache\derbyTesting\functionTests\tests\memoryIO".

-----

Thanks.


      was (Author: chatom):
    Hi folks,

Attached (derby-646-20090222.stat, derby-646-20090222.diff) is my implementation of the interfaces
org.apache.derby.io.{StorageFile,StorageRandomAccessFile,WritableStorageFactory} such that
all file system operations are emulated entirely in memory, producing no file or directory
on disk.

A brief description of the content:

- java\engine\org\apache\derby\impl\io\memory

  This directory contains the source files for the implementation of the database storage
factory interfaces.

- java\testing\org\apache\derbyTesting\functionTests\tests\memoryIO

  This directory contains the source files for the JUnit tests.  It consists of three sets
of fairly extensive unit tests of the basic building blocks of my implementation.  They started
out as five sets of stand-alone JUnit tests and were converted to conform to Derby's JUnit
test framework.  In the process of conversion, two sets of stand-alone tests were dropped
because they use on-disk operations to verify the correctness of in-memory operations, and
Derby's test framework does not allow arbitrary operations on disk, as far as I can tell.
 These unit tests run only in the embedded mode of Derby's test framework, because they make
no DB connections.

  In addition, there is one functional test consisting of a simple but non-trivial clustering
application.  It is non-trivial because clustering is an important application in many engineering
and scientific disciplines; it is simple in that it computes clustering of numerical data
in one dimensional space -- the simplest case.  The application implements the same algorithm
in both JAVA (computed by the JUnit test case object itself) and SQL (computed by Derby using
the in-memory storage factory) and cross-checks the computed results, within some floating-point
tolerance.  The SQL portion of the application computes quite a few CREATE, INSERT, DELETE,
SELECT, JOIN, GROUP BY, ORDER BY, DROP; and so should exercise the storage layer to a good
extent.  This functional test runs in both the embedded mode and the client mode of Derby's
test framework.

The sub-sub-protocol associated with this in-memory storage factory is named "memory".  So
the connection string looks like "jdbc:derby:memory:MyTestDb" or "jdbc:derby://localhost:1527/memory:MyTestDb".

-----

HELP!!!  I've gotten the tests compiled properly with "ant all".  The compiled class files
show up under "classes\org\apache\derbyTesting\functionTests\tests\memoryIO".  But again I'm
running into the problem of not knowing how to get them included in the "derbyTesting.jar"
file.

Nor did I figure out how to have them automatically executed with "ant junit-all", "ant junitreport",
etc.  For now, this is what I do after "ant all" to execute the tests on Windows:

  java -cp .\tools\java\junit.jar;.\classes junit.textui.TestRunner org.apache.derbyTesting.functionTests.tests.memoryIO._Suite

-----

Existing files modified by this patch:

- java\engine\org\apache\derby\impl\build.xml
  tools\jar\extraDBMSclasses.properties

  These two files were modified so that my implementation would be compiled and included in
Derby's JAR files.

- java\engine\org\apache\derby\iapi\services\monitor\PersistentService.java
  java\engine\org\apache\derby\impl\services\monitor\BaseMonitor.java
  
  These two files were modified only because I did not feel like typing "-Dderby.subSubProtocol.memory=org.apache.derby.impl.io.memory.MemoryStorageFactory"
for every run.  I simply added a mapping from the "memory" service to the "org.apache.derby.impl.io.memory.MemoryStorageFactory"
class in BaseMonitor.

- java\testing\build.xml

  This file was modified so that my tests would be compiled into class files under "classes\org\apache\derbyTesting\functionTests\tests\memoryIO".

-----

Thanks.

  
> In-memory backend storage support
> ---------------------------------
>
>                 Key: DERBY-646
>                 URL: https://issues.apache.org/jira/browse/DERBY-646
>             Project: Derby
>          Issue Type: New Feature
>          Components: Store
>         Environment: All
>            Reporter: Stephen Fitch
>         Attachments: derby-646-1a-raw-compiles.diff, derby-646-1a-raw-compiles.stat,
derby-646-20090222.diff, derby-646-20090222.stat, derby-646-2a-vfmem_first_rev.diff, derby-646-2a-vfmem_first_rev.stat,
svn.diff
>
>
> To allow creation and modification of databases in-memory without requiring disk access
or space to store the database.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message