db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajesh Kartha <kar...@Source-Zone.Org>
Subject Re: proposed modification current test harness (Re: DERBY-575)
Date Wed, 12 Oct 2005 20:42:56 GMT
Kathey Marsden wrote:

>Daniel John Debrunner wrote:
>[snip conversation about how to
>
>  
>
>> 
>>
>>the counter is with your solution how do they generate a
>>master file suitable for submitting as part of a contribution? I'm not
>>sure there is a good solution.
>>
>> 
>>
>>    
>>
>I see nothing wrong with leaving test development on OS390 as another
>fish to fry,  and  I think in general Myrna's plan is a good one if I
>understand it correctly.   Generally we agree on an encoding for master
>files, sql files etc and always check in files in that encoding.  Then
>the corresponding files under the test output directory are always in
>the native encoding making tests easy to run and failures easy to
>diagnose. 
>
>To me what is a little fuzzy to me is the agreed input encoding for the
>sql and master files, especially for languages like Japanese that are
>not going to conform to ISO 8859-1.  What would seem to make sense to me
>would be ISO 8859-1 with escape sequences like the common  property file
>format  in Java
>(http://java.sun.com/j2se/1.4.2/docs/api/java/util/Properties.html#load(java.io.InputStream).

>That way you could run native2ascii on whatever platform you were
>working with if you had special  characters in the text to get the
>desired master file.  Does specifying an input encoding  of ISO 8859-1 
>read in the escaped  sequences properly, or do you need to specify
>something else? 
>
>Kathey
>
>
>  
>
I was thinking, for test development on platforms like OS/390 etc. - 
that has different encoding, wouldn't  it help if the user adds the 
master encoding  information
as one of the test properties (for example: a property like  - 
derby.test.master.encoding). He/she should be able to then submit the 
master (in its orginal encoding)
without any need for conversion. The harness can  get this property and 
use it to read the master and if needed convert them to 'UTF-8' to be 
readable
on regular platforms or skip the test. I would think the same will work 
for  files in Korean, Japanese etc.

Something like:
http://java.sun.com/docs/books/tutorial/i18n/text/stream.html

For the reasons pointed out by Myrna, I would agree the harness needs to 
convert the master (from the testing jar) to the local encoding for the 
users to understand
the failures. I would also like to add that instead of writing the 
converted files to the local disk, if the master can the converted to 
the local encoding using
streams and be kept in the memory (streams).  The following snippet uses 
'*UTF-8*' .

For example: on  zOS


                    //read the default master file from the Derby 
testing jar

                    InputStream aStream2 = new FileInputStream(infile2);

                      //UTF-8 being used as default encoding for reading 
the master file.

                       Reader aReader2 = new 
InputStreamReader(aStream2,"UTF-8");
                        BufferedReader br2=new BufferedReader(aReader2);

                     //convert to local encoding while writing to the 
outputstream

                        ByteArrayOutputStream bao=new 
ByteArrayOutputStream();
                        PrintWriter eWriter = new PrintWriter(bao);
                        String s=null;
                        while((s = br2.readLine())!=null) {
                               eWriter.println(s);
                        }
                        eWriter.close();

                     //get the new converted input stream

                       ByteArrayInputStream bio=new 
ByteArrayInputStream(bao.toByteArray());
                        aReader2 = new InputStreamReader(bio);
                        br2=new BufferedReader(aReader2);
                    
                     //do comparision between the newly converted master 
and the actual output

This will avoid the writing of the converted files to the disk and hence 
the need for cleanup as originally suggested.
Moreover this conversion needs to* happen only if *the 
System.getPropertty ("file.encoding") is of a special type, say 
Cp1047(zOS) etc.

My 2 cents,

-Rajesh







Mime
View raw message