Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 13 Apr 2017 08:06:41 +0000 (UTC)
From: "Duo Zhang (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.13063714.1492067861000.273825.1492070801688@Atlassian.JIRA>
In-Reply-To: <JIRA.13063714.1492067861000@Atlassian.JIRA>
References: <JIRA.13063714.1492067861000@Atlassian.JIRA> <JIRA.13063714.1492067861042@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HBASE-17910) Use separated StoreFileReader for
 streaming read
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 13 Apr 2017 08:06:46 -0000


    [ https://issues.apache.org/jira/browse/HBASE-17910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15967268#comment-15967268 ] 

Duo Zhang commented on HBASE-17910:
-----------------------------------

I do not think this modification will have a big impact on the number of active file handlers? In most cases we will use pread I think.

And anpther problem is the connection number to datanode, this is a problem for the dfs client, we need to introduce a new multiplex protocol to communicate with datanode.

> Use separated StoreFileReader for streaming read
> ------------------------------------------------
>
>                 Key: HBASE-17910
>                 URL: https://issues.apache.org/jira/browse/HBASE-17910
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Duo Zhang
>
> For now we have already supportted using private readers for compaction, by creating a new StoreFile copy. I think a better way is to allow creating multiple readers from a single StoreFile instance, thus we can avoid the ugly cloning, and the reader can also be used for streaming scan, not only for compaction.
> The reason we want to do this is that, we found a read amplification when using short circult read. {{BlockReaderLocal}} will use an internal buffer to read data first, the buffer size is based on the configured buffer size and the readahead option in CachingStrategy. For normal pread request, we should just bypass the buffer, this can be achieved by setting readahead to 0. But for streaming read I think the buffer is somehow still useful? So we need to use different FSDataInputStream for pread and streaming read.
> And one more thing is that, we can also remove the streamLock if streaming read always use its own reader.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)