db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Deepa Remesh (JIRA)" <derby-...@db.apache.org>
Subject [jira] Updated: (DERBY-683) Use correct encoding for ClobOutputStream on client
Date Fri, 03 Feb 2006 04:17:11 GMT
     [ http://issues.apache.org/jira/browse/DERBY-683?page=all ]

Deepa Remesh updated DERBY-683:

    Attachment: derby-683_tests.diff

I am attaching a patch 'derby-683_tests.diff' which adds testing for this issue. The patch
changes the harness to enable running tests/suites using an encoding different from the default
system encoding. 

To test DERBY-683, I wanted to run an existing test (jdbcapi/lobStreams.java) using an encoding
which will have different representation for ASCII characters. I found that UTF-16 uses more
number of bits and different representation for ASCII characters than the US-ASCII encoding.
Also, UTF-16 is in the list of standard charsets supported by all jvms specified at http://java.sun.com/j2se/1.5.0/docs/api/java/nio/charset/Charset.html

For this, I first tried using 'jvmflags' property in the test harness. Harness uses this property
to launch the jvm process to run tests. So it is possible to do this only when useprocess=true.
I could run the test by specifying "jvmflags=-Dfile.encoding=UTF-16" in <test>_app.properties
file. However, the test output gets written using UTF-16 encoding and it appears in strange
format. To read the output, harness needs to know what encoding the test jvm used. For this,
it has to parse the jvmflags, look for "-Dfile.encoding=", get and store the encoding specified,
launch the jvm using the encoding, and later use that encoding to read the test output. Since
jvmflags property is used by harness for other purposes, I added a new property 'derbyTesting.encoding'
to the harness. Harness will read this property to get the encoding and internally change
jvmflags before launching the jvm to run tests. As mentioned before, all this is possible
only when useprocess=true. 

I don't think my changes cover all the things that harness needs to support to run tests with
different encodings. So I am trying to list what we can/cannot do with these changes: 
* Can specify the encoding in <test>_app.properties file and run an individual test
(not as part of a suite). If the encoding property is specified for a test, it gets used only
if the test is run individually using RunTest. When the test gets run as part of a suite,
useprocess is set to false in RunTest and no new jvm gets launched for the test. So the encoding
property won't get used.
* Can specify the encoding in <suite>.properties file and run the whole suite of tests
using that encoding. In this case, a new jvm is launched for RunTest class and all tests are
run with the encoding specified. 

* Cannot successfully run sql tests if the encoding property is specified. The sql files will
get read using the specified encoding and may not be meaningful. If the sql files always get
read using fixed encoding, this will not be a problem. I think this change is planned in DERBY-658
by Myrna.	

* Can run tests/suites using encoding property and look for possible areas for cleanup. 

Once the patch is reviewed and if it is okay, I will update the testing readme file with this

This patch does the following: 

* Adds a new property 'derbyTesting.encoding' to the test harness. This property can be specified
as a suite in <suite>.properties file or as test property in <test>_app.properties
file. If this property is set at suite level, it overrides the property set in the tests inside
that suite. For this, I made changes to RunSuite.java, RunList.java and RunTest.java to read
the property at each level only if it is not set at the higher level. 

* The patch uses the value specified in 'derbyTesting.encoding=<enc_value>' for two

1. To append to the jvmflags property used to start the child jvm process. Following is appended:
-Dfile.encoding=<enc_value>. Currently, I have hard-coded the prefix "-Dfile.encoding="
in the code. In case any new jvm uses a different property name, the prefix can be specified
for each jvm in the
corresponding jvm class and this can be retrieved and used.
2. To read the test output: 
 - when using RunTest, ProcessStreamResult class is used to read the output of the jvm process
and write to the output file. The patch changes this class to use InputStreamReader which
is created using <enc_value> encoding. The OutputStreamWriter which writes to .out file
is created using system default encoding. This needs to use the default encoding because the
harness will compare the .out file to master using <default_enc>.
- when using RunSuite, HandleResult class is used to read the test output. The patch changes
this class to use InputStreamReader which is created using <enc_value> encoding. 

* Creates a new suite encodingTests with jdbcapi/lobStreams.java in .runall file. I created
a new suite because I cannot specify the encoding property in a test and run it as part of
a suite. 

* Adds derbyTesting.encoding=UTF-16 to encodingTests.properties file. Adds excludes for all
jvms except Sun jdk1.5. 

* Adds the encodingTests suite to derbynetclientmats

With this patch, I ran derbyall with Sun JDK1.4.2 on Windows XP. No failures. With Sun JDK1.5,
I ran the new suite encodingTests and verified that the specified encoding is getting used
to run the test. 
Please review this patch. Thanks

> Use correct encoding for ClobOutputStream on client
> ---------------------------------------------------
>          Key: DERBY-683
>          URL: http://issues.apache.org/jira/browse/DERBY-683
>      Project: Derby
>         Type: Bug
>   Components: Network Client
>     Versions:,
>  Environment: all
>     Reporter: Sunitha Kambhampati
>     Assignee: Deepa Remesh
>      Fix For:
>  Attachments: ascii.txt, clob.java, derby-683.diff, derby-683_tests.diff, derby-683_tests.status
> In client, there is code in ClobOutputStream which uses this api - new String(byte[]).
  Per the java api http://java.sun.com/j2se/1.4.2/docs/api/java/lang/String.html#String(byte[])
,this will construct a string by decoding the array of bytes using the platform's default
character set. 
> org.apache.derby.client.am.ClobOutputStream is used for Clob.setAsciiStream and the write
methods  use the String(byte[]) which is incorrect because it will use the default platform
encoding. Per the jdbcapi , this should use ascii encoding. 
> In areas related to Clobs, also check for other places where  String(byte[]) is used,as
it may not be the desired behavior. 
> Dan pointed this problem here : http://issues.apache.org/jira/browse/DERBY-463?page=comments#action_12356742

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators:
For more information on JIRA, see:

View raw message