hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15811) Batch Get after batch Put does not fetch all Cells
Date Thu, 12 May 2016 17:52:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281834#comment-15281834
] 

stack commented on HBASE-15811:
-------------------------------

Here is a hack that won't work for general case -- executor is not exclusive to HTable always
-- but that I'm currently testing with... 

{code}
 1 diff --git a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java
  2 index fb46365..56b8d67 100644
  3 --- a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java
  4 +++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java
  5 @@ -193,6 +193,9 @@ class AsyncProcess {
  6    protected final ExecutorService pool;
  7
  8    protected final AtomicLong tasksInProgress = new AtomicLong(0);
  9 +  public AtomicLong getTasksInProgress() {
 10 +      return this.tasksInProgress;
 11 +  }
 12    protected final ConcurrentMap<byte[], AtomicInteger> taskCounterPerRegion =
 13        new ConcurrentSkipListMap<byte[], AtomicInteger>(Bytes.BYTES_COMPARATOR);
 14    protected final ConcurrentMap<ServerName, AtomicInteger> taskCounterPerServer
=
 15 diff --git a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/BufferedMutatorImpl.java
b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/BufferedMutatorI    mpl.java
 16 index 6220cd6..6a21d80 100644
 17 --- a/hbase-client/src/main/java/org/apache/hadoop/hbase/client/BufferedMutatorImpl.java
 18 +++ b/hbase-client/src/main/java/org/apache/hadoop/hbase/client/BufferedMutatorImpl.java
 19 @@ -253,6 +253,10 @@ public class BufferedMutatorImpl implements BufferedMutator {
 20          dequeuedSize -= size;
 21          writeAsyncBuffer.add(mut);
 22        }
 23 +      while (ap.getTasksInProgress().get() > 0) {
 24 +          LOG.info("TASKS IN PROGRESS " + ap.getTasksInProgress().get());
 25 +          org.apache.hadoop.hbase.util.Threads.sleep(1);
 26 +      }
 27      }
 28    }
{code}

Seems to work which would mean we've figured the issue here. Let me do some more extensive
testing.

> Batch Get after batch Put does not fetch all Cells
> --------------------------------------------------
>
>                 Key: HBASE-15811
>                 URL: https://issues.apache.org/jira/browse/HBASE-15811
>             Project: HBase
>          Issue Type: Bug
>          Components: Client
>    Affects Versions: 1.2.1
>            Reporter: stack
>            Assignee: stack
>            Priority: Blocker
>         Attachments: Test.java, Test2.java
>
>
> A big batch put followed by a batch get does not always return all Cells put. See attached
test program by Robert Farr that reproduces the issue. It seems to be an issue to do with
a cluster of more than one machine. Running against a single machine does not have the problem
(though the single machine may have many regions). Robert was unable to make his program fail
with a single machine only.
> I reproduced what Robert was seeing running his program. I was also unable to make a
single machine fail. In a batch of 1000 puts, I see one to three Gets fail. I noticed too
that if I wait a second after a fail and then re-get, the Get succeeds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message