hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12782) ITBLL fails for me if generator does anything but 5M per maptask
Date Fri, 30 Jan 2015 06:28:34 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14298268#comment-14298268
] 

stack commented on HBASE-12782:
-------------------------------

This seems to be the fix:

{code}
diff --git a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
index 87ebebb..bddb5f5 100644
--- a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
+++ b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
@@ -3793,9 +3796,7 @@ public class HRegion implements HeapSize, PropagatingConfigurationObserver
{ //
             // Once we are over the limit, restoreEdit will keep returning true to
             // flush -- but don't flush until we've played all the kvs that make up
             // the WALEdit.
-            if (!flush) {
-              flush = restoreEdit(store, cell);
-            }
+            flush = restoreEdit(store, cell);

             editsCount++;
           }
{code}

Would match what I am seeing where the first few cells of a big edit make it in but the tail
of the edit does not.  Whenever we came up to the flushing limit replaying recovered edits,
we'd start skipping the application of edits though comment says we're adding them.

Let me get some runs in in the morning to try and get confirmation this is it.

> ITBLL fails for me if generator does anything but 5M per maptask
> ----------------------------------------------------------------
>
>                 Key: HBASE-12782
>                 URL: https://issues.apache.org/jira/browse/HBASE-12782
>             Project: HBase
>          Issue Type: Bug
>          Components: integration tests
>    Affects Versions: 1.0.0
>            Reporter: stack
>            Priority: Critical
>             Fix For: 1.0.1
>
>         Attachments: 12782.search.plus.archive.recovered.edits.txt, 12782.search.plus.txt,
12782.search.txt, 12782.unit.test.and.it.test.txt, 12782.unit.test.writing.txt
>
>
> Anyone else seeing this?  If I do an ITBLL with generator doing 5M rows per maptask,
all is good -- verify passes. I've been running 5 servers and had one splot per server.  So
below works:
> HADOOP_CLASSPATH="/home/stack/conf_hbase:`/home/stack/hbase/bin/hbase classpath`" ./hadoop/bin/hadoop
--config ~/conf_hadoop org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList --monkey
serverKilling Generator 5 5000000 g1.tmp
> or if I double the map tasks, it works:
> HADOOP_CLASSPATH="/home/stack/conf_hbase:`/home/stack/hbase/bin/hbase classpath`" ./hadoop/bin/hadoop
--config ~/conf_hadoop org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList --monkey
serverKilling Generator 10 5000000 g2.tmp
> ...but if I change the 5M to 50M or 25M, Verify fails.
> Looking into it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message