hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Lawlor (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13262) ResultScanner doesn't return all rows in Scan
Date Tue, 17 Mar 2015 18:36:38 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365731#comment-14365731
] 

Jonathan Lawlor commented on HBASE-13262:
-----------------------------------------

Good catch! Took a quick look and it seems to be related to the change in the default caching
value from HBASE-11544 when coupled with the change in the default max result size from HBASE-12976.
I was actually able to reproduce this issue in branch-1.0 by tweeking the caching and max
result size to be equal to the defaults that are present in branch-1+. Specifically, the following
configuration also produced issues for me in branch-1.0:

{code}
Scan scan = new Scan();
scan.setMaxResultSize(2 * 1024 * 1024);
scan.setCaching(Integer.MAX_VALUE);
...
{code}

(Note that to run the scan with this configuraiton you must remove the ArrayList presizing
that is performed in RSRpcServices on line 2055. The caching value is used to presize the
array and thus you will likely run into OOME if you do not remove the presizing when running
this configuration). 

As to why this error is occuring, I am not sure yet, but I thought this may shed some light
on it.

> ResultScanner doesn't return all rows in Scan
> ---------------------------------------------
>
>                 Key: HBASE-13262
>                 URL: https://issues.apache.org/jira/browse/HBASE-13262
>             Project: HBase
>          Issue Type: Bug
>          Components: Client
>    Affects Versions: 2.0.0, 1.1.0
>         Environment: Single node, pseduo-distributed 1.1.0-SNAPSHOT
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>            Priority: Blocker
>             Fix For: 2.0.0, 1.1.0
>
>
> Tried to write a simple Java client again 1.1.0-SNAPSHOT.
> * Write 1M rows, each row with 1 family, and 10 qualifiers (values [0-9]), for a total
of 10M cells written
> * Read back the data from the table, ensure I saw 10M cells
> Running it against {{04ac1891}} (and earlier) yesterday, I would get ~20% of the actual
rows. Running against 1.0.0, returns all 10M records as expected.
> [Code I was running|https://github.com/joshelser/hbase-hwhat/blob/master/src/main/java/hbase/HBaseTest.java]
for the curious.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message