hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4334) HRegion.get never validates row
Date Mon, 03 Oct 2011 18:17:41 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119458#comment-13119458
] 

Lars Hofhansl commented on HBASE-4334:
--------------------------------------

There's a thread on the mailing about a similar issue:
we are experiencing a strange behavior in some tests we are currently performing. What we
are seeing is that scans on a table that is being written to at the same time sometimes end
prematurely, with no error. This seems to be heavily dependent on the write pattern.

We've been able to reproduce the issue with the standard hbase tools:

in a terminal run:

OLDV=0; OLDT=0; while true; do NEWV=`hbase shell count_testtable | head -1 | cut -d' ' -f1`;
NEWT=`date +%s`; echo $NEWV " -> " $(((NEWV - OLDV) / (NEWT - OLDT))) "msg/s"; OLDV=$NEWV;
OLDT=$NEWT; done

where the contents of the file count_testtable are:

count 'TestTable', INTERVAL => 100000000, CACHE => 10000
exit

This counts the rows in the TestTable repeatedly showing the number of rows and the delta
rows per second inserted. In the hbase shell count is implemented as a full scan with a filter
on the row key.

Meanwhile, in another terminal do:

hbase org.apache.hadoop.hbase.PerformanceEvaluation --nomapred --rows=20000 randomWrite 5

and when over:

hbase org.apache.hadoop.hbase.PerformanceEvaluation --nomapred --rows=20000 sequentialWrite
5

On the scan terminal we are seeing results similar to:

0  ->  0 msg/s
45552  ->  5694 msg/s <=== randomWrite starts
63284  ->  2955 msg/s
63284  ->  0 msg/s  <=== randomWrite ends
58829  ->  -636 msg/s <=== sequentialWrite starts
88764  ->  3741 msg/s
100000  ->  802 msg/s
100000  ->  0 msg/s <=== sequentialWrite ends

As you can see in the fifth row the count is lower than expected (resulting on a negative
inserts/sec).

You may need to try a couple of times or tweak the number of rows to insert to see the problem.

                
> HRegion.get never validates row
> -------------------------------
>
>                 Key: HBASE-4334
>                 URL: https://issues.apache.org/jira/browse/HBASE-4334
>             Project: HBase
>          Issue Type: Bug
>          Components: regionserver
>    Affects Versions: 0.90.4
>            Reporter: Joe Pallas
>            Assignee: Lars Hofhansl
>             Fix For: 0.92.0
>
>         Attachments: 4334-v2.txt, 4334-v3.txt, 4334.txt
>
>
> If a client gets confused (possibly by a hole in .META., see HBASE-4333), it may send
a request to the wrong region.  Paths through put, delete, incrementColumnValue, and checkAndMutate
all call checkRow either directly or indirectly (through getLock).  But get apparently does
not.  This can result in returning an incorrect empty result instead of a WrongRegionException.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message