hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bwolen Yang" <wbwo...@gmail.com>
Subject Re: \r\n problem in LineRecordReader.java
Date Wed, 13 Jun 2007 15:35:37 GMT
On 6/13/07, Raghu Angadi <rangadi@yahoo-inc.com> wrote:
> Bwolen Yang wrote:
> > Here is probably the cause of this bug:
> >
> >    public int read(byte b[], int off, int len) throws IOException {
> >      // make sure that it ends at a checksum boundary
> >      long curPos = getPos();
> >      long endPos = len+curPos/bytesPerSum*bytesPerSum;
> >      return readBuffer(b, off, (int)(endPos-curPos));
> >    }
> >
> > Here, the caller calls the function with 127 bytes, and bytesPerSum is 256.
>
> Is this from looking at the code or you actually saw the values like
> this at runtime?

taking values at runtime (i have it thru exceptions when the result is
0 and print out he values).

> I think 'len' is never supposed to be less than bytesPerChecksum because
> there is a BufferedInputStream between with a buffer size of
> io.buffer.size (default 4096). So this buffer size is supposed to be
> larger than bytesPerChecksum (requirement changes with HADOOP-1450 where
> it uses buffer size equal to bytesPerChecksum).

bwolen

Mime
View raw message