Mailing-List: contact harmony-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: harmony-dev@incubator.apache.org
Received-SPF: neutral (asf.osuosl.org: 217.158.94.220 is neither permitted nor
 denied by domain of t.p.ellison@gmail.com)
Message-ID: <440306CE.4000606@gmail.com>
Date: Mon, 27 Feb 2006 14:03:58 +0000
From: Tim Ellison <t.p.ellison@gmail.com>
User-Agent: Thunderbird 1.5 (Windows/20051201)
MIME-Version: 1.0
To: harmony-dev@incubator.apache.org
Subject: Re: [jira] Commented: (HARMONY-137) CharsetDecoder should replace
 undefined bytes with replacement string
References: <2072939177.1141029896480.JavaMail.jira@ajax.apache.org>
In-Reply-To: <2072939177.1141029896480.JavaMail.jira@ajax.apache.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Paulex,

Please add the ICU bug number to this JIRA when you have it -- thanks.

Regards,
Tim

Paulex Yang (JIRA) wrote:
>     [ http://issues.apache.org/jira/browse/HARMONY-137?page=comments#action_12367917 ] 
> 
> Paulex Yang commented on HARMONY-137:
> -------------------------------------
> 
> A little investigation on the Harmony codes, seems it is caused by problems of ICU4JNI decoder provider, the following test cases shows that.  RI "cp1250" passes the testcase while ICU "cp1250" fails under either RI or Harmony . I'll try to report the bug to ICU.
> 
> Test case:
> 
> 	public void testDecode_JIRA137() {
> 		ByteBuffer bb = ByteBuffer.allocate(5);
> 		bb.put((byte) 0x81);
> 		bb.flip();
> 		// Use ICU cp1250 charset
> 		CharsetProviderICU provider = new CharsetProviderICU();
> 		Charset cp1250 = provider.charsetForName("cp1250");
> 		// Uncomment code below to use RI charset
> 		//cp1250 = Charset.forName("cp1250");
> 		CharBuffer cb;
> 		try {
> 			cb = cp1250.newDecoder()
> 					.onMalformedInput(CodingErrorAction.REPLACE)
> 					.onUnmappableCharacter(CodingErrorAction.REPLACE)
> 					.decode(bb);
> 			assertEquals(0XFFFD,cb.get(0));
> 		} catch (CharacterCodingException e) {
> 			e.printStackTrace();
> 		}
> 	}
> 
>> CharsetDecoder should replace undefined bytes with replacement string
>> ---------------------------------------------------------------------
>>
>>          Key: HARMONY-137
>>          URL: http://issues.apache.org/jira/browse/HARMONY-137
>>      Project: Harmony
>>         Type: Bug
>>   Components: Classlib
>>     Reporter: Vladimir Strigun
>>     Priority: Minor
> 
>> Corresponding to cp1250 mapping table, 0x81 byte is undefined. See http://www.unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1250.TXT
>> So, charset decoder should replace undefined bytes with default replacement, i.e. 0xFFFD. 
>> Testcase for reproducing this issue:
>> import java.nio.charset.*;
>> import java.nio.*;
>> public class Harmony137 {
>>     public static void main(String[] args) throws Exception {
>>         ByteBuffer bb = ByteBuffer.allocate(5);
>>         bb.put((byte)0x81); bb.flip();
>>         Charset cp1250 = Charset.forName("cp1250");
>>         CharBuffer cb = cp1250.newDecoder().onMalformedInput(CodingErrorAction.REPLACE).onUnmappableCharacter(CodingErrorAction.REPLACE).decode(bb);
>>         if(cb.get(0)!=65533) {
>>             System.out.println("FAIL: expected 0xFFFD but result is: 0x"+Integer.toHexString(cb.get(0)).toUpperCase());
>>         }
>>     }
>> }
> 

-- 

Tim Ellison (t.p.ellison@gmail.com)
IBM Java technology centre, UK.