hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chia-Ping Tsai (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-18158) Two running in-memory compaction threads may lose data
Date Sun, 04 Jun 2017 15:49:04 GMT
Chia-Ping Tsai created HBASE-18158:

             Summary: Two running in-memory compaction threads may lose data
                 Key: HBASE-18158
                 URL: https://issues.apache.org/jira/browse/HBASE-18158
             Project: HBase
          Issue Type: Bug
    Affects Versions: 2.0.0
            Reporter: Chia-Ping Tsai
             Fix For: 2.0.0

  private void stopCompaction() {
    if (inMemoryFlushInProgress.get()) {
The stopCompaction() set inMemoryFlushInProgress to false so there may be two in-memory compaction
thread are executed simultaneously. If there are two running InMemoryFlushRunnable, the later
InMemoryFlushRunnable may change the versionedList. 
  public boolean start() throws IOException {
    if (!compactingMemStore.hasImmutableSegments()) { // no compaction on empty pipeline
      return false;

    // get a snapshot of the list of the segments from the pipeline,
    // this local copy of the list is marked with specific version
    versionedList = compactingMemStore.getImmutableSegments();
And the first InMemoryFlushRunnable will use the chagned versionedList to remove the corresponding
      if (!isInterrupted.get()) {
        if (resultSwapped = compactingMemStore.swapCompactedSegments(
            versionedList, result, (action==Action.MERGE))) {
          // update the wal so it can be truncated and not get too long
          compactingMemStore.updateLowestUnflushedSequenceIdInWAL(true); // only if greater
In conclusion, first InMemoryFlushRunnable will remove the worng segment. And the later InMemoryFlushRunnable
will introduce NPE because first InMemoryFlushRunnable set versionedList to null after compaction.
Exception in thread "RpcServer.default.FPBQ.Fifo.handler=3,queue=0,port=45712-inmemoryCompactions-1496563908038"
        at org.apache.hadoop.hbase.regionserver.CompactionPipeline.swap(CompactionPipeline.java:119)
        at org.apache.hadoop.hbase.regionserver.CompactingMemStore.swapCompactedSegments(CompactingMemStore.java:283)
        at org.apache.hadoop.hbase.regionserver.MemStoreCompactor.doCompaction(MemStoreCompactor.java:212)
        at org.apache.hadoop.hbase.regionserver.MemStoreCompactor.start(MemStoreCompactor.java:122)
        at org.apache.hadoop.hbase.regionserver.CompactingMemStore.flushInMemory(CompactingMemStore.java:388)
        at org.apache.hadoop.hbase.regionserver.CompactingMemStore$InMemoryFlushRunnable.run(CompactingMemStore.java:500)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

This message was sent by Atlassian JIRA

View raw message