hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14307) Incorrect use of positional read api in HFileBlock
Date Wed, 26 Aug 2015 17:50:47 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14715223#comment-14715223

Chris Nauroth commented on HBASE-14307:

[~ram_krish] and [~anoop.hbase], thank you for your replies.

Just to add a little more context on Shradha's information, the contract of [{{FSDataInputStream}}|http://hadoop.apache.org/docs/r2.7.1/api/org/apache/hadoop/fs/PositionedReadable.html]
guarantees that it will read "up to the specified number of bytes".  It does not guarantee
that it will read exactly that number of bytes.  Even if the backing file contains that many
bytes remaining, a file system implementation may choose to return immediately after reading
any bytes that are already available in buffer space, without forcing the caller to block
waiting for the exact number of bytes to be available.

This is similar to the contract of [{{java.io.InputStream}}|http://docs.oracle.com/javase/7/docs/api/java/io/InputStream.html],
which states that "a smaller number may be read".

bq. In many places we use IOUtils.readFully API which in turn will use loop. I dont think
there is some reason why we should not use that here as well.

That sounds like exactly the kind of change we had in mind.  It sounds like the intent of
this logic in HBase is to detect premature EOF, not necessarily demand that a single read
call completes the entire operation.

Thank you both!

> Incorrect use of positional read api in HFileBlock
> --------------------------------------------------
>                 Key: HBASE-14307
>                 URL: https://issues.apache.org/jira/browse/HBASE-14307
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Shradha Revankar
>            Priority: Minor
> Considering that {{read()}} is not guaranteed to read all bytes, 
> I'm interested to understand this particular piece of code and why is partial read treated
as an error :
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java#L1446-L1450
> Particularly, if hbase were to use a different filesystem, say WebhdfsFileSystem, this
would not work, please also see https://issues.apache.org/jira/browse/HDFS-8943 for discussion
around this.

This message was sent by Atlassian JIRA

View raw message