hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4487) The increment operation can release the rowlock before sync-ing the Hlog
Date Thu, 29 Sep 2011 22:31:48 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13117677#comment-13117677
] 

jiraposter@reviews.apache.org commented on HBASE-4487:
------------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2116/#review2186
-----------------------------------------------------------


Looks good. I like the bench marking tool.

This is like a group commit in HLog.  We have a group committing going on inside in sync down
in dfsclient.  Does this group commit not 'group' enough.

With this change, its no longer possible to sync each increment.  We ok w/ that?  We should
at least release note this difference in increment behaviors and erring on the conservative
side, I'd mark this an 'incompatible change' rather than an 'improvement' just so its more
likely folks will know of this changed increment behavior.

- Michael


On 2011-09-29 19:27:36, Dhruba Borthakur wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/2116/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2011-09-29 19:27:36)
bq.  
bq.  
bq.  Review request for hbase and Ted Yu.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  The increment operation releases the rowlock before doing the sync to the HLog. This
improves performance of increments on hot rows. Introduced method HLog.appendNoSync() that
returns a txid. The increment method then release the rowlock and invokes HLog.sync(txid).
The HLog.sync(txid) returns only if all the transactions upto the one identified by that txid
has been successfully sycned to HDFS.
bq.  
bq.  There is a single test TestIncrement that creates a 100 threads and ensures that all
the concurrent increments match the final expected result. There is a benchmark TestHLogBench
that measures the performance of the appendNoSync call.
bq.  
bq.  
bq.  This addresses bug HBASE-4487.
bq.      https://issues.apache.org/jira/browse/HBASE-4487
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    /src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1177401 
bq.    /src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 1177401 
bq.    /src/test/java/org/apache/hadoop/hbase/regionserver/TestIncrement.java PRE-CREATION

bq.    /src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogBench.java PRE-CREATION

bq.    /src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALActionsListener.java
1177401 
bq.    /src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java
1177401 
bq.  
bq.  Diff: https://reviews.apache.org/r/2116/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  All unit tests pass now (expect TestDistributedLogSplitting, TestRollingRestart, TestHTablePool),
but I am seeing the same test to fail on trunk, so these failures do not seem to be related
to this patch.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Dhruba
bq.  
bq.


                
> The increment operation can release the rowlock before sync-ing the Hlog
> ------------------------------------------------------------------------
>
>                 Key: HBASE-4487
>                 URL: https://issues.apache.org/jira/browse/HBASE-4487
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>         Attachments: appendNoSync4.txt, appendNoSync5.txt, appendNoSync6.txt
>
>
> This allows for better throughput when there are hot rows.I have seen this change make
a single row update improve from 400 increments/sec/server to 4000 increments/sec/server.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message