hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raghu Angadi <rang...@yahoo-inc.com>
Subject Re: Problem with InputStream.skip()
Date Thu, 31 May 2007 18:59:28 GMT
Better one:

// There are differences in how skip and seek behave
// when trying to skip more bytes than available in a file
// but ...
long skip(long len)  throws IOException {
    if ( len > 0 ) {
      seek(getPos() + len);
      return len;
    }
    return ( len < 0 ) ? -1 : 0;
}

This should fix HADOOP-1428, if not very efficiently.

Raghu.

Raghu Angadi wrote:
> 
> We should force subclasses of FSInputStream to implement skip if skip is 
>  expected to be used. Only way I could think of achieving is to define
> 
> long skip(long len)  throws IOException {
>    throw new IOException("Subclasses of FSInputStream should implement 
> skip");
> }
> 
> This makes sense for all the current FSInputStreams. But not sure if 
> there is any way for subclasses to call InputStream.skip() is that makes 
> sense.
> 
> Raghu.
> 
> Doug Cutting wrote:
>> Raghu Angadi wrote:
>>> Also, reading from block supports 'real skip', ie, it does not check 
>>> checksum if an entire checksum block (usually 512 bytes) falls within 
>>> the skip range. Another reason to implement our own skip.
>>
>> Yes, I don't see an alternative to implementing skip ourselves.  The 
>> optimization in InputStream#skip(), of using a static buffer, requires 
>> this.  Hopefully this method, like much of the checksum code, will be 
>> shared between the generic ChecksumFileSystem and DFS's optimized 
>> checksum implementation.
>>
>> Doug
> 


Mime
View raw message