hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allan Yang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-17471) Region Seqid will be out of order in WAL if using mvccPreAssign
Date Sun, 15 Jan 2017 16:15:26 GMT
Allan Yang created HBASE-17471:

             Summary: Region Seqid will be out of order in WAL if using mvccPreAssign
                 Key: HBASE-17471
                 URL: https://issues.apache.org/jira/browse/HBASE-17471
             Project: HBase
          Issue Type: Bug
          Components: wal
    Affects Versions: 2.0.0, 1.4.0
            Reporter: Allan Yang
            Assignee: Allan Yang

 mvccPreAssign was bring by HBASE-16698, which truly improved the performance of writing,
especially in ASYNC_WAL scenario. But mvccPreAssign was only used in {{doMiniBatchMutate}},
not in Increment/Append path. If Increment/Append and batch put are using against the same
region in parallel, then seqid of the same region may not monotonically increasing in the
WAL. Since one write path acquires mvcc/seqid before append, and the other acquires in the
append/sync consume thread.

The out of order situation can easily reproduced by a simple UT, which was attached in the
attachment. I modified the code to assert on the disorder: 
    if(this.highestSequenceIds.containsKey(encodedRegionName)) {
      assert highestSequenceIds.get(encodedRegionName) < sequenceid;

I'd like to say, If we are allow disorder in WALs, then this is not a issue. 

But as far as I know, if {{highestSequenceIds}} is not properly set, some WALs may not archive
to oldWALs correctly.

which I haven't figure out yet is that, will disorder in WAL cause data loss when recovering
from disaster? If so, then it is a big problem need to be fixed.

I have fix this problem in our costom1.1.x branch, my solution is using mvccPreAssign everywhere,
making it un-configurable. Since mvccPreAssign it is indeed a better way than assign seqid
in the ringbuffer thread while keeping handlers waiting for it.

If anyone think it is doable, then I will port it to branch-1 and master branch and upload


This message was sent by Atlassian JIRA

View raw message