harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paulex Yang <paulex.y...@gmail.com>
Subject Re: [classlib] charset decoding
Date Wed, 26 Apr 2006 13:53:05 GMT
Andrew Zhang wrote:
>
> Mikhail Loenko wrote:
>> 2006/4/26, Andrew Zhang <zhanghuangzhu@gmail.com>:
>>> Vladimir Strigun wrote:
>>>> On 4/26/06, Andrew Zhang <zhanghuangzhu@gmail.com> wrote:
>>>> I try to find best solution for Harmony-166 and during fix preparation
>>>> I've found current issue.
>>>>
>>>> Method test_read from tests.api.java.io.InputStreamReaderTest failed
>>>> and first fix I've created was with in.available() but I agree with
>>>> Mikhail that it's possible not the best solution.
>>> why?
>>>
>>> Yes, the spec says "The available method for class InputStream always
>>> returns 0.". But it also says "This method should be overridden by
>>> subclasses.". :)
>>>
>>> Here's the available description:
>>> "Returns the number of bytes that can be read (or skipped over) from
>>> this input stream without blocking by the next caller of a method for
>>> this input stream."
>>>
>>> If someone writes a subclass of InputStream that available returns 0
>>> even if there are some data available, then he should take the 
>>> result of
>>> cheating or contradicting with spec.
>
>>
>> What if someone just unable to say if the bytes are available?
>> For example, he reads something from a hardware module and
>> the only way to know whether there are bytes is try reading?
>>
>
> I'm a little confused.
>
> How does solution 2 handle this problem?
I agree with Mikhail that available is not reliable.  You can judge the 
stream end has been reached if and only if you got -1 returned from read 
methods.

I did a quick view of Harmony-166 and patches(seems it's the root of why 
Harmony-410 is raised), I think there should be some easier way to fix 
this bug,  while the CharsetDecoder doesn't need to be modified, pls. 
see below for details.

The fillBuf() of InputStreamReader should be looks like:

    private void fillBuf() throws IOException {
        chars.clear();
        int read = 0;
       // if we haven't reached the end of stream, and cannot decode 
even one character,
      // go on to read and decode
        while(read != -1 && chars.position() == 0){
            try {
                read = in.read(bytes.array());
            } catch (IOException e) {
                chars.limit(0);
                throw e;
            }
            boolean endOfInput = false;
            if(read == -1){
                //if we have reached the end of stream, try the last 
time to decode
                bytes.limit(0);
                endOfInput = true;
            }else{
                //if we read some bytes, try to decode them
                bytes.limit(read);
                read = 0;
            }
            decoder.decode(bytes, chars, endOfInput);
            bytes.clear();
        }
        chars.flip();
    }

I've got a pass to run Vladimir's test case for Harmony-166.

If no one objections, I'll attach patch based on this. comments?
>
>> Thanks,
>> Mikhail
>>
>>>> In current implementation of InputStreamReader endOfInput variable
>>>> sets to true if reader can't read less that 8192 bytes. When
>>>> InputStreamReader try to read one character from
>>>> LimitedByteArrayInputStream (A ByteArrayInputStream that only returns
>>>> a single byte per read) true as a parameter passed to charset decoder,
>>>> nevertheless we still have further input from
>>>> LimitedByteArrayInputStream.
>>>>
>>>> 2 methods from tests.api.java.io.OutputStreamWriterTest also failed
>>>> because of read() method implementation of InputStreamReader.
>>>>
>>>> IMO, we have 2 ways to fix Harmony-166 :
>>>>
>>>> 1. Don't change the behavior of CharsetDecoder, and use in.available()
>>>> method for endOfInput variable. If in.available() > 0 pass false to
>>>> decoder, read additional one byte from input stream and try to decode
>>>> again. If decoding operation can decode remaining portion + one
>>>> additional byte to character, stop here, else try to read addition
>>>> byte again. Function fillBuf will invoke with the next invocation of
>>>> read() method and next portion of 8192 bytes will be read from input
>>>> stream.
>>>>
>>> It's reasonable.
>>> After all, streaming decode is useful and helpful to users.
>>> IMO, streaming decode is a highlight compared with some RI :)
>>>
>>>> 2. Change the behavior of CharsetDecoder and operate with
>>>> bytes.remaining() for calculating endOfInput. In this case, we always
>>>> pass false with first invocation of decode method. Algorithm further
>>>> is almost the same as above, but we stop the cycle if all bytes
>>>> decoded successfully (i.e if bytes.hasRemaining() is false).
>
> Vladimir, Could you please give the detail code or pseudo code of 
> these two solutions?
>
> Thanks!
>
>>>>
>>>> What do you think about it?
>>>>
>>>>
>>>> Thanks.
>>>> Vladimir.
>>> Thanks!
>
>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>


-- 
Paulex Yang
China Software Development Lab
IBM



---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message