hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon" <t...@cloudera.com>
Subject Re: Review Request: Batch writes should sync to HLog in batches
Date Fri, 11 Jun 2010 22:18:10 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated 2010-06-11 15:18:10.087969)

Review request for hbase, Kannan Muthukkaruppan and Ryan Rawson.


Significantly changed patch - now batch put returns a list of status codes. Also added some
tests to make sure all the behavior is correct with locking, bad families, etc.

I'd like to change HRegionInterface and HTable to return lists of status codes at some point
too, but I think that can wait for a later patch. This one at least gets our performance back
up by batching the syncs.


I implemented the "mini batching" idea we talked about on the JIRA.

This currently breaks some of the error handling, so I dont intend to commit as is, but everyone
is busy so wanted to put a review up now while I tidy up the rest.

This addresses bug HBASE-2353.

Diffs (updated)

  src/main/java/org/apache/hadoop/hbase/HConstants.java 1e59533 
  src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 62617ac 
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java adc505b 
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 541ec9b 
  src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java ba04820 
  src/test/java/org/apache/hadoop/hbase/MultithreadedTestUtil.java 870f925 
  src/test/java/org/apache/hadoop/hbase/TestAcidGuarantees.java 8a5206c 
  src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java e75767d 

Diff: http://review.hbase.org/r/167/diff


Some PEs on a real sync-enabled cluster, seems faster but haven't done scientific benchmarking.



View raw message