db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Army <qoz...@sbcglobal.net>
Subject [PATCH] DERBY-121: Network Server reading blob/clob data size.
Date Thu, 02 Jun 2005 18:18:29 GMT
[ http://issues.apache.org/jira/browse/DERBY-121 ]

Attached is a patch to fix the bit-shift error described in DERBY-121.  The 
error exists both in the Network Server code and in the Derby Client code.

The fix, which is described in the description of the JIRA entry, is to include 
a bit-shift for bits 24 thru 31 of a lob's length.  Before this patch, bits 0 
thru 23 were shifted, and then bits 32 thru <max> were shifted, but bits 24 thru 
31 were excluded.

The problem with the old code only shows up with lobs that have a declared 
length that is represented by at least 24 bits--in other words, the lob must 
have a length of at least 2^24 bytes (2^23 chars for a clob).

The fix itself is straightforward, but the catch was in creating a _test_.  In 
order to reproduce the problem and then verify the fix, a test must create a LOB 
that is at least 2^24 bytes long and pass that on to the server.  In order for 
that to succeed, the server JVM must be started with more memory than usual--in 
my own experience, the server required "-ms128M -mx128M" in order to correctly 
handle the test case.

Since every Derby contributor is asked to run the "derbyall" suite before 
submitting a patch, and since we don't want to require that all such 
contributors have a machine with lots of spare memory, I did NOT want to put the 
new test for DERBY-121 into the "derbyall" suite.  Instead, I decided to work 
off the notion described by Mike Matrigali in the following email:

http://thread.gmane.org/gmane.comp.apache.db.derby.devel/4713

In a word, Mike suggested that we create a new test suite (separate from 
derbyall) intended for tests that require "large resources".  We would NOT 
include this new suite in "derbyall", nor would we ask every Derby contributor 
to run it.  Instead, we would hope that someone could volunteer the 
machines/resources required to run the new suite at some regular interval (not 
yet determined).

So, since Mike brought that up, and since the situation applies to the test that 
I wrote for DERBY-121, I went ahead and created a generic "largeData" suite that 
contains my test for DERBY-121.  I then created a properties file for the new 
test and, in that file, specified (using the "jvmflags" property) that the test 
requires at least "-mx128M -ms128M".  And finally, I made a small change in the 
harness/NetServer.java file so that it recognizes if heap size properties were 
specified by the test and, if so, it uses them instead of the default server 
heap size.

-- Patch details: The new suite.

For the new suite, I tried to create one that mimics other suites in the 
harness--but as I've never created a suite before, I hope someone with more 
knowledge will review the suite (ex. Myrna, do you have time? ;)

To create the new suite and include my test for DERBY-121, I had to add the 
following files:

- suites/largeDataTests.runall: A list of all tests that should be included in 
the new "largeData" suite.  Right now this just has one entry--the test for 
DERBY-121.

- suites/largeData.properties: The new, top-level suite.  Runs all tests listed 
in largeDataTests.runall, and then calls two subsuites for running with the 
DerbyNet and DerbyNetClient frameworks.

- suites/largeDataNet.properties: A new subsuite that runs all tests listed in 
largeDataTests.runall, but specifies that they should all be run with the 
"DerbyNet" framework.

- suites/largeDataClient.properties: A new subsuite that runs all tests listed 
in largeDataTests.runall, but specifies that they should all be run with the 
"DerbyNetClient" framework.

- tests/largedata/lobLengthTests.java: A new test written for DERBY-121.  I've 
tried to make the test generic enough that other cases related to the lengths of 
very large LOBs can be added in the future (if needed).

- tests/largedata/lobLengthTests_app.properties: A properties file for the new 
test.  This file sets the "jvmflags" property so that the server JVM will start 
with a sufficient heap size to complete the test.

- tests/largedata/copyfiles.ant: A file required so that the *_app.properties 
files will be copied to the build path at compile time.  I based this on files 
of the same name in different test directories (ex. tests/jdbcapi).

- tests/largedata/build.xml: A build file for the new suite.  I have to admit 
that I'm not entirely sure how necessary this is, but since all of the other 
test directories had a similar file (ex. tests/jdbcapi), I copied one of them 
and made changes where I thought appropriate.  If this file is unnecessary, or 
if the contents need to be altered, could someone please tell me? (and in 
particular, Andrew (fuzzylogic), can you take a look at this?)

** This new suite is NOT meant to completely satisfy the requirements listed by 
Mike in his email above.  Rather, this is just a starting point--something that 
can hopefully be extended/expanded in the future to more closely fulfill the 
functionality that Mike was asking about (at least, the "large resources" 
functionality; the idea of long-running tests may require its own suite...)

-- Other patch info.

In order to apply the patch, you may need to create the following directory first:

java/testing/org/apache/derbyTesting/functionTests/tests/largedata

I've successfully built and ran the new suite (using harness.RunSuite), and have 
also run the new test in standalone mode (using harness.RunTest), and everything 
ran correctly.  I then ran the full "derbyall" suite to make sure my changes 
didn't break anything: on a Windows 2000 machine with Sun JDK 1.4.2, all tests 
passed (except for those that are known issues, unrelated to my changes).

I know there are a lot of patches out there for review right now, but since this 
patch (namely, the new "largeData" suite) could have an impact on future 
tests/patches in this area, I'm hoping some people can review this and let me 
know if I need to change anything...

Feedback is appreciated,
Army

Mime
View raw message