Return-Path: X-Original-To: apmail-hbase-dev-archive@www.apache.org Delivered-To: apmail-hbase-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id DA09317897 for ; Fri, 17 Apr 2015 14:46:59 +0000 (UTC) Received: (qmail 59821 invoked by uid 500); 17 Apr 2015 14:46:59 -0000 Delivered-To: apmail-hbase-dev-archive@hbase.apache.org Received: (qmail 59657 invoked by uid 500); 17 Apr 2015 14:46:58 -0000 Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hbase.apache.org Delivered-To: mailing list dev@hbase.apache.org Received: (qmail 59500 invoked by uid 99); 17 Apr 2015 14:46:58 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 17 Apr 2015 14:46:58 +0000 Date: Fri, 17 Apr 2015 14:46:58 +0000 (UTC) From: "ChiaPing Tsai (JIRA)" To: dev@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Resolved] (HBASE-13492) The estimation of result size may be different between ClientScanner and RSRpcServices MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-13492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ChiaPing Tsai resolved HBASE-13492. ----------------------------------- Resolution: Duplicate > The estimation of result size may be different between ClientScanner and RSRpcServices > -------------------------------------------------------------------------------------- > > Key: HBASE-13492 > URL: https://issues.apache.org/jira/browse/HBASE-13492 > Project: HBase > Issue Type: Bug > Affects Versions: 1.0.0 > Reporter: ChiaPing Tsai > > The ClientScanner try to find next scanner if remainingResultSize and countdown are bigger than zero. > The remainingResultSize is calculated by CellUtil.estimatedHeapSizeOf(cell) > {code:title=Bar.java|borderStyle=solid} > @Override > public Result next() throws IOException { > .... > do { > ... > if (values != null && values.length > 0) { > for (Result rs : values) { > cache.add(rs); > // We don't make Iterator here > for (Cell cell : rs.rawCells()) { > remainingResultSize -= CellUtil.estimatedHeapSizeOf(cell); > } > countdown--; > this.lastResult = rs; > } > } > }while (remainingResultSize > 0 && countdown > 0 && > possiblyNextScanner(countdown, values == null)); > } > {code} > RSRpcServices also use CellUtil.estimatedHeapSizeOf(cell) to calculate the result size > {code:title=Bar.java|borderStyle=solid} > @Override > public ScanResponse scan(final RpcController controller, final ScanRequest request) > throws ServiceException { > ... > if (!results.isEmpty()) { > for (Result r : results) { > for (Cell cell : r.rawCells()) { > currentScanResultSize += CellUtil.estimatedHeapSizeOf(cell); > totalCellSize += CellUtil.estimatedSerializedSizeOf(cell); > } > } > } > ... > } > {code} > If we encode the data block, like FastDiff, the cell read from HFile is implemented by ClonedSeekerState. And it's heap size is bigger than KeyValue. > So the RSRpcServices return the results to client with insufficient caching due to result size reaches the limit. ClientScanner consider that current region has no more data, and remainingResultSize and countdown are both bigger than zero. In fact, the remainingResultSize should be smaller than zero, and current region still have more data for reading. > Does result size calculated by RSRpcServices should be return to client for checking the remainingResultSize ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)