hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "rahul gidwani (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-8806) Row locks are acquired repeatedly in HRegion.doMiniBatchMutation for duplicate rows.
Date Tue, 25 Jun 2013 19:50:20 GMT
rahul gidwani created HBASE-8806:

             Summary: Row locks are acquired repeatedly in HRegion.doMiniBatchMutation for
duplicate rows.
                 Key: HBASE-8806
                 URL: https://issues.apache.org/jira/browse/HBASE-8806
             Project: HBase
          Issue Type: Bug
          Components: regionserver
    Affects Versions: 0.94.5
            Reporter: rahul gidwani
             Fix For: 0.98.0, 0.95.0, 0.94.0

If we already have the lock in the doMiniBatchMutation we don't need to re-acquire it. The
solution would be to keep a cache of the rowKeys already locked for a miniBatchMutation and
If we already have the 
rowKey in the cache, we don't repeatedly try and acquire the lock.  A fix to this problem
would be to keep a set of rows we already locked and not try to acquire the lock for these

We have tested this fix in our production environment and has improved replication performance
quite a bit.  We saw a replication batch go from 3+ minutes to less than 10 seconds for batches
with duplicate row keys.

static final int ACQUIRE_LOCK_COUNT = 0;

  public void testRedundantRowKeys() throws Exception {

    final int batchSize = 100000;
    String tableName = getClass().getSimpleName();
    Configuration conf = HBaseConfiguration.create();
    conf.setClass(HConstants.REGION_IMPL, MockHRegion.class, HeapSize.class);
    MockHRegion region = (MockHRegion) TestHRegion.initHRegion(Bytes.toBytes(tableName), tableName,
conf, Bytes.toBytes("a"));

    List<Pair<Mutation, Integer>> someBatch = Lists.newArrayList();
    int i = 0;
    while (i < batchSize) {
      if (i % 2 == 0) {
        someBatch.add(new Pair<Mutation, Integer>(new Put(Bytes.toBytes(0)), null));
      } else {
        someBatch.add(new Pair<Mutation, Integer>(new Put(Bytes.toBytes(1)), null));
    long startTime = System.currentTimeMillis();
    region.batchMutate(someBatch.toArray(new Pair[0]));
    long endTime = System.currentTimeMillis();
    long duration = endTime - startTime;
    System.out.println("duration: " + duration + " ms");
    assertEquals(2, ACQUIRE_LOCK_COUNT); <--- should only try and acquire the lock twice.

  public Integer getLock(Integer lockid, byte[] row, boolean waitForLock) throws IOException
    return super.getLock(lockid, row, waitForLock);

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message