Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@harmony.apache.org
Received-SPF: pass (herse.apache.org: domain of alexey.a.petrenko@gmail.com
 designates 64.233.184.230 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=beta;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
        b=YU0TXc5xh0FfJXIgXZsPURSIafw9GYGDGIyr1QZs1FeI9OPclev/cSik4nkYsMmtxXiGeQJjdtVx+PNtR3Fi9QXORj4pkXSS0nN3Qiim05+8lpBRh9I4RtBHghqa5GQy6CztbH+Se3ycDadrbwswHGuV0lp92crFgTdaFWoR1D8=
Message-ID: <c3755b3a0704240353n2b890c4btbe8d8174126736ea@mail.gmail.com>
Date: Tue, 24 Apr 2007 14:53:39 +0400
From: "Alexey Petrenko" <alexey.a.petrenko@gmail.com>
To: dev@harmony.apache.org
Subject: Re: [vm]different system property: file.encoding
In-Reply-To: <f944d83d0704240339p2bf198b0j549ac64d44ebdce7@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <211709bc0704200224w2c854684xdeecfd78abbbeb05@mail.gmail.com>
	 <f944d83d0704200516t86e76f5odadd7b35cadbed1d@mail.gmail.com>
	 <211709bc0704230317r2a9bea44h15df5731b7d38056@mail.gmail.com>
	 <c3755b3a0704232213i1e76bfaqea086ecea4afbc17@mail.gmail.com>
	 <f944d83d0704232221q6aeb786cu28932c13f86cfce2@mail.gmail.com>
	 <f944d83d0704232346x2463e700vfddf24cfd3e061fc@mail.gmail.com>
	 <c3755b3a0704240107l7f75edabgc61f5ef376b3cd30@mail.gmail.com>
	 <f944d83d0704240339p2bf198b0j549ac64d44ebdce7@mail.gmail.com>

I think we should wait till next week anyway. For stability reasons...

SY, Alexey

2007/4/24, Alexey Varlamov <alexey.v.varlamov@gmail.com>:
> As it appears, IBM VME only supports small predefined array of
> properties and does not allow to add custom ones via VMI. Not sure if
> this limitation is a bug or feature of IBM VME...
> A workaround is to ignore failed attempts to set such properties, I've
> attached corrected patch.
>
> 2007/4/24, Alexey Petrenko <alexey.a.petrenko@gmail.com>:
> > Unfortunately IBM VME fails with your patch while starting unit tests :(
> > Here is error message:
> >
> > run-tests:
> >     [echo] Running LUNI JUnit Tests
> >    [mkdir] Created dir: C:\Users\cap\harmony\working_classlib\build\test_report
> >    [junit] <error: unable to load hyluni (Incompatible JNI version
> > (not 1.1, 1.2 or 1.4))>
> >    [junit] JVMJ9VM015W Initialization error for library
> > jclclear_23(14): JVMJ9VM009E J9VMDllMain failed
> >    [junit] HMYEXEL062E Internal VM error: Failed to create Java VM
> >    [junit] FAILED to invoke JVM.
> >    [junit] java.io.FileNotFoundException:
> > C:\Users\cap\harmony\working_classlib\modules\luni\junitvmwatcher2005985557.properties:
> > The system cannot find the file specified
> >    [junit]     at
> > jrockit.io.FileNativeIO.open0(Ljava.lang.String;I)I(Native Method)
> >    [junit]     at
> > jrockit.io.FileNativeIO.open(Ljava.lang.String;I)Ljava.io.FileDescriptor;(Unknown
> > Source)
> >    [junit]     at
> > java.io.FileInputStream.<init>(Ljava.lang.String;)V(Unknown Source)
> >    [junit]     at java.io.FileInputStream.<init>(Ljava.io.File;)V(Unknown Source)
> >    [junit]     at java.io.FileReader.<init>(FileReader.java:55)
> >    [junit]     at
> > org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.executeAsForked(JUnitTask.java:1028)
> >    [junit]     at
> > org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.execute(JUnitTask.java:859)
> >    [junit]     at
> > org.apache.tools.ant.taskdefs.optional.junit.JUnitTask.execute(JUnitTask.java:782)
> >    [junit]     at
> > org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:288)
> >    [junit]     at
> > jrockit.reflect.VirtualNativeMethodInvoker.invoke(Ljava.lang.Object;[Ljava.lang.Object;)Ljava.lang.Object;(Unknown
> > Source)
> >    [junit]     at
> > java.lang.reflect.Method.invoke(Ljava.lang.Object;[Ljava.lang.Object;I)Ljava.lang.Object;(Unknown
> > Source)
> >    [junit]     at
> > org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:105)
> >    [junit]     at org.apache.tools.ant.Task.perform(Task.java:348)
> >    [junit]     at
> > org.apache.tools.ant.taskdefs.Sequential.execute(Sequential.java:62)
> >    [junit]     at
> > org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:288)
> >    [junit]     at
> > jrockit.reflect.VirtualNativeMethodInvoker.invoke(Ljava.lang.Object;[Ljava.lang.Object;)Ljava.lang.Object;(Unknown
> > Source)
> >    [junit]     at
> > java.lang.reflect.Method.invoke(Ljava.lang.Object;[Ljava.lang.Object;I)Ljava.lang.Object;(Unknown
> > Source)
> >    [junit]     at
> > org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:105)
> >    [junit]     at org.apache.tools.ant.Task.perform(Task.java:348)
> >    [junit]     at
> > org.apache.tools.ant.taskdefs.MacroInstance.execute(MacroInstance.java:391)
> >    [junit]     at
> > org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:288)
> >    [junit]     at
> > jrockit.reflect.VirtualNativeMethodInvoker.invoke(Ljava.lang.Object;[Ljava.lang.Object;)Ljava.lang.Object;(Unknown
> > Source)
> >    [junit]     at
> > java.lang.reflect.Method.invoke(Ljava.lang.Object;[Ljava.lang.Object;I)Ljava.lang.Object;(Unknown
> > Source)
> >    [junit]     at
> > org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:105)
> >    [junit]     at org.apache.tools.ant.Task.perform(Task.java:348)
> >    [junit]     at org.apache.tools.ant.Target.execute(Target.java:357)
> >    [junit]     at org.apache.tools.ant.Target.performTasks(Target.java:385)
> >    [junit]     at
> > org.apache.tools.ant.Project.executeSortedTargets(Project.java:1329)
> >    [junit]     at
> > org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
> >    [junit]     at org.apache.tools.ant.Project.executeTargets(Project.java:1181)
> >    [junit]     at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:416)
> >    [junit]     at org.apache.tools.ant.taskdefs.SubAnt.execute(SubAnt.java:289)
> >    [junit] Running tests.api.java.util.WeakHashMapTest
> >    [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
> >
> >
> > 2007/4/24, Alexey Varlamov <alexey.v.varlamov@gmail.com>:
> > > FYI, I've created HARMONY-3736 and attached patch creating
> > > placeholders for properties setup.
> > >
> > > 2007/4/24, Alexey Varlamov <alexey.v.varlamov@gmail.com>:
> > > > 2007/4/24, Alexey Petrenko <alexey.a.petrenko@gmail.com>:
> > > > > Looks like we need a JIRA for DRLVM...
> > > >
> > > > This issue is about classlib impl details, we should minimize burden
> > > > of classlib support for VM. Please see below.
> > > >
> > > > >
> > > > > SY, Alexey
> > > > >
> > > > > 2007/4/23, Tony Wu <wuyuehao@gmail.com>:
> > > > > > Thanks Alexey,
> > > > > > I'm not sure whether it could be set up in classlib. This property is
> > > > > > used by String.getBytes(), which is called when ClassLoader initial.
> > > > > > If we want to set up a value before ClassLoader initial, we have to do
> > > > > > in vm (kernel class). Am I right?
> > > >
> > > > This should be done in natives initialization, which is performed
> > > > before any Java code is executed; see JNI_OnLoad() in
> > > > working_classlib\modules\luni\src\main\native\luni\shared\luniglob.c
> > > >
> > > > > >
> > > > > > And no matter where we decide to do that at last, we need the mapping
> > > > > > data from locale to encoding. How to get and store is a problem too.
> > > > > >
> > > > > > On 4/20/07, Alexey Varlamov <alexey.v.varlamov@gmail.com> wrote:
> > > > > > > 2007/4/20, Tony Wu <wuyuehao@gmail.com>:
> > > > > > > > Hi all,
> > > > > > > >
> > > > > > > > There is a difference in default system property - "file.encoding". It
> > > > > > > > comes from vm[1]. I considerd it was a non-bug difference because
> > > > > > > > there isn't any description in spec. What's your opinion?
> > > > > > > >
> > > > > > > > [1]
> > > > > > > > e.g. for locale zh_CN, the output
> > > > > > > > IBM VM: GB18030
> > > > > > > > DRLVM: 8859-1
> > > > > > > > RI        : GBK
> > > > > > > >
> > > > > > > > 3307 and 3702 are related issues.
> > > > > > >
> > > > > > > There are several open entries in SUN's bug parade for documenting
> > > > > > > "extra" properties, typical resolution says [1]:
> > > > > > > "Users are discouraged from any reliance on implementation details.
> > > > > > > The key "file.encoding" in particular is an implementation artifact
> > > > > > > that may not be defined or used by other vendors."
> > > > > > >
> > > > > > > But since we rely on this property in *our implementation*, we need to
> > > > > > > perform flawlessly - putting aside the difference with RI.
> > > > > > > DRLVM simply hardcodes this property value, no any runtime detection
> > > > > > > is performed. I think it would be nice if classlib could set up
> > > > > > > appropriate value at startup (new JIRA needed?).
> > > > > > >
> > > > > > > [1] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6334530
> > > > > > >
> > > > > > > >
> > > > > > > > ---------- Forwarded message ----------
> > > > > > > > From: Vasily Zakharov (JIRA) <jira@apache.org>
> > > > > > > > Date: Apr 20, 2007 2:46 PM
> > > > > > > > Subject: [jira] Commented: (HARMONY-3702) [classlib][luni] Reader and
> > > > > > > > Writer convert characters incorrectly
> > > > > > > > To: commits@harmony.apache.org
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >   [ https://issues.apache.org/jira/browse/HARMONY-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12490241
> > > > > > > > ]
> > > > > > > >
> > > > > > > > Vasily Zakharov commented on HARMONY-3702:
> > > > > > > > ------------------------------------------
> > > > > > > >
> > > > > > > > I'm not sure if this is a non-bug difference. Probably we should have
> > > > > > > > the same default encoding as RI, for compatibility.
> > > > > > > >
> > > > > > > > Probably this should be discussed in the mailing list.
> > > > > > > >
> > > > > > > >
> > > > > > > > > [classlib][luni] Reader and Writer convert characters incorrectly
> > > > > > > > > -----------------------------------------------------------------
> > > > > > > > >
> > > > > > > > >                 Key: HARMONY-3702
> > > > > > > > >                 URL: https://issues.apache.org/jira/browse/HARMONY-3702
> > > > > > > > >             Project: Harmony
> > > > > > > > >          Issue Type: Bug
> > > > > > > > >          Components: Classlib
> > > > > > > > >            Reporter: Vasily Zakharov
> > > > > > > > >         Attachments: test.dat
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > java.io.Reader converts bytes to characters differently than RI does. Also, java.io.Writer converts characters to bytes differently than RI does.
> > > > > > > > > The attached test.dat file contains random test data and must be placed to the current directory. ReaderTest below reads that file with FileReader and then dumps it to standard output by converting each character to int. WriterTest reads the test.dat file with FileInputStream, converts each byte to character by casting and then dumps the resulting characters to standard output by OutputStreamWriter.
> > > > > > > > > public class ReaderTest {
> > > > > > > > >     public static void main(String args[]) throws Exception {
> > > > > > > > >         char[] buffer = new char[0x100000];
> > > > > > > > >         java.io.Reader reader = new java.io.FileReader("test.dat");
> > > > > > > > >         int length = reader.read(buffer, 0, buffer.length);
> > > > > > > > >         for (int i = 0; i < length; i++) {
> > > > > > > > >             System.out.println((int) buffer[i]);
> > > > > > > > >         }
> > > > > > > > >     }
> > > > > > > > > }
> > > > > > > > > public class WriterTest {
> > > > > > > > >     public static void main(String args[]) throws Exception {
> > > > > > > > >         byte[] buffer = new byte[0x100000];
> > > > > > > > >         java.io.InputStream iStream = new java.io.FileInputStream("test.dat");
> > > > > > > > >         int length = iStream.read(buffer, 0, buffer.length);
> > > > > > > > >         char[] charBuffer = new char[length];
> > > > > > > > >         for (int i = 0; i < length; i++) {
> > > > > > > > >             charBuffer[i] = (char) buffer[i];
> > > > > > > > >         }
> > > > > > > > >         java.io.Writer writer = new java.io.OutputStreamWriter(System.out);
> > > > > > > > >         writer.write(charBuffer, 0, length);
> > > > > > > > >         writer.close();
> > > > > > > > >     }
> > > > > > > > > }
> > > > > > > > > In both cases, output files on RI and on Harmony are different:
> > > > > > > > > $ RI/bin/java ReaderTest > reader.ri
> > > > > > > > > $ HY/bin/java ReaderTest > reader.hy
> > > > > > > > > $ diff --binary -q reader.ri reader.hy
> > > > > > > > > Files reader.ri and reader.hy differ
> > > > > > > > > $ RI/bin/java WriterTest > writer.ri
> > > > > > > > > $ HY/bin/java WriterTest > writer.hy
> > > > > > > > > $ diff --binary -q writer.ri writer.hy
> > > > > > > > > Files writer.ri and writer.hy differ
> > > > > > > > > My investigations show that the problem is in Reader/Writer, not in InputStream/OutputStream. Also, I've tried other implementations of Reader/Writer and they share the same problem.
> > > > > > > > > The problem was discovered on Windows XP/IA-32 but probably affects other platforms too.
> > > > > > > >
> > > > > > > > --
> > > > > > > > This message is automatically generated by JIRA.
> > > > > > > > -
> > > > > > > > You can reply to this email to add a comment to the issue online.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Tony Wu
> > > > > > > > China Software Development Lab, IBM
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Tony Wu
> > > > > > China Software Development Lab, IBM
> > > > > >
> > > > >
> > > >
> > >
> >
>