Mailing-List: contact harmony-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: harmony-dev@incubator.apache.org
Message-ID: <90895006.1141116220981.JavaMail.jira@ajax.apache.org>
Date: Tue, 28 Feb 2006 09:43:40 +0100 (CET)
From: "Richard Liang (JIRA)" <jira@apache.org>
To: harmony-dev@incubator.apache.org
Subject: [jira] Commented: (HARMONY-137) CharsetDecoder should replace
 undefined bytes with replacement string
In-Reply-To: <1964685948.1141024287236.JavaMail.jira@ajax.apache.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

    [ http://issues.apache.org/jira/browse/HARMONY-137?page=comments#action_12368087 ] 

Richard Liang commented on HARMONY-137:
---------------------------------------

Please see the bug info in ICU bug system: http://bugs.icu-project.org/cgi-bin/icu-bugs?findid=5085&go=Go

And attached here is ICU team's response to this bug:

You are expecting incorrect behavior from cp1250. Both Microsoft's conversion APIs and IBM mapping tables convert byte 81 to Unicode character 0081. This conversion behavior will not change. The tables on unicode.org may tell you about the official mappings, but there are other mappings that are commonly expected.

More details about ICU charset conversion can be found on this page: http://icu.sourceforge.net/charts/charset/

This charset conversion works as expected.


> CharsetDecoder should replace undefined bytes with replacement string
> ---------------------------------------------------------------------
>
>          Key: HARMONY-137
>          URL: http://issues.apache.org/jira/browse/HARMONY-137
>      Project: Harmony
>         Type: Bug
>   Components: Classlib
>     Reporter: Vladimir Strigun
>     Priority: Minor

>
> Corresponding to cp1250 mapping table, 0x81 byte is undefined. See http://www.unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1250.TXT
> So, charset decoder should replace undefined bytes with default replacement, i.e. 0xFFFD. 
> Testcase for reproducing this issue:
> import java.nio.charset.*;
> import java.nio.*;
> public class Harmony137 {
>     public static void main(String[] args) throws Exception {
>         ByteBuffer bb = ByteBuffer.allocate(5);
>         bb.put((byte)0x81); bb.flip();
>         Charset cp1250 = Charset.forName("cp1250");
>         CharBuffer cb = cp1250.newDecoder().onMalformedInput(CodingErrorAction.REPLACE).onUnmappableCharacter(CodingErrorAction.REPLACE).decode(bb);
>         if(cb.get(0)!=65533) {
>             System.out.println("FAIL: expected 0xFFFD but result is: 0x"+Integer.toHexString(cb.get(0)).toUpperCase());
>         }
>     }
> }

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira