hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Lawlor (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-13374) Small scanners (with particular configurations) do not return all rows
Date Wed, 01 Apr 2015 20:45:54 GMT

     [ https://issues.apache.org/jira/browse/HBASE-13374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Lawlor updated HBASE-13374:
------------------------------------
    Attachment: HBASE-13374-v1.patch

Attaching a patch that fixes both issues. If the start key is set correctly on the small scanner
callable then the caching does not need to be incremented (thus avoiding the integer overflow)
and skipRowOfFirstResult can be removed.

> Small scanners (with particular configurations) do not return all rows
> ----------------------------------------------------------------------
>
>                 Key: HBASE-13374
>                 URL: https://issues.apache.org/jira/browse/HBASE-13374
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 1.0.0, 2.0.0, 1.1.0, 0.98.13
>            Reporter: Jonathan Lawlor
>            Assignee: Jonathan Lawlor
>            Priority: Blocker
>             Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.13
>
>         Attachments: HBASE-13374-v1.patch, small-scanner-data-loss-tests-0.98.patch,
small-scanner-data-loss-tests-branch-1.0+.patch
>
>
> I recently ran into a couple data loss issues with small scans. Similar to HBASE-13262,
these issues only appear when scans are configured in such a way that the max result size
limit is reached before the caching limit is reached. As far as I can tell, this issue affects
branches 0.98+
> I should note that after investigation it looks like the root cause of these issues is
not the same as HBASE-13262. Rather, these issue are caused by errors in the small scanner
logic (I will explain in more depth below). 
> Furthermore, I do know that the solution from HBASE-13262 has not made its way into small
scanners (it is being addressed in HBASE-13335). As a result I made sure to test these issues
with the patch from HBASE-13335 applied and I saw that they were still present.
> The following two issues have been observed (both lead to data loss):
> 1. When a small scan is configured with a caching value of Integer.MAX_VALUE, and a maxResultSize
limit that is reached before the region is exhausted, integer overflow will occur. This eventually
leads to a preemptive skip of the regions.
> 2. When a small scan is configured with a maxResultSize that is smaller than the size
of a single row, the small scanner will jump between regions preemptively. This issue seems
to be because small scanners assume that, unless a region is exhausted, at least 2 rows will
be returned from the server. This assumption isn't clearly state in the small scanners but
is implied through the use of {{skipRowOfFirstResult}}.
> Again, I would like to stress that the root cause of these issues is *NOT* related to
the cause of HBASE-13262. These issues occur because of inappropriate assumption made in the
small scanner logic. The inappropriate assumptions are:
> 1. Integer overflow will not occur when incrementing caching
> 2. At least 2 rows will be returned from the server unless the region has been exhausted
> I am attaching a patch that contains tests to display these issues. If these issues should
be split into separate JIRAs please let me know.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message