hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon" <t...@cloudera.com>
Subject Re: Review Request: HBASE-1923. Bulk load into existing tables
Date Sun, 30 May 2010 23:13:22 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/87/
-----------------------------------------------------------

(Updated 2010-05-30 16:13:22.509910)


Review request for hbase, stack and Jonathan Gray.


Changes
-------

Here's an updated diff which includes Stack's comments, plus does the junk inside Store to
get rid of the currentTimeMillis hack. Also augments the test case a bit to force the bulk-loaded
regions to reload.

Still needs further work, and some new unit tests, again just want to get early comments since
this has become rather invasive.


Summary
-------

Here's a first patch that implements bulk import into existing tables. This applies on top
of HBASE-2586 and HBASE-2588 - I've pushed the series of the three to my github: http://github.com/toddlipcon/hbase/tree/hfof-review

I have some TODOs left that I want to take care of before this gets committed, but since it's
a pretty large patch, I figured I'd get it out for review ASAP.

The stuff in the hadoopbackport package is essentially copypaste from Hadoop trunk, so you
can ignore that in the review.


This addresses bug HBASE-1923.
    http://issues.apache.org/jira/browse/HBASE-1923


Diffs (updated)
-----

  pom.xml 0a009cf 
  src/main/java/org/apache/hadoop/hbase/HRegionInfo.java 29b0cd6 
  src/main/java/org/apache/hadoop/hbase/io/ImmutableBytesWritable.java 0a9ec4b 
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java b912a85 
  src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 4cbe52a 
  src/main/java/org/apache/hadoop/hbase/mapreduce/Driver.java 3d40695 
  src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java 9c8e53e 
  src/main/java/org/apache/hadoop/hbase/mapreduce/ImportTsv.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java PRE-CREATION

  src/main/java/org/apache/hadoop/hbase/mapreduce/PutSortReducer.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/mapreduce/SimpleTotalOrderPartitioner.java af3d588

  src/main/java/org/apache/hadoop/hbase/mapreduce/hadoopbackport/InputSampler.java PRE-CREATION

  src/main/java/org/apache/hadoop/hbase/mapreduce/hadoopbackport/TotalOrderPartitioner.java
PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 287cd48 
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 78f3223 
  src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 7de766d 
  src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java 80bf09a 
  src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java c8941f1 
  src/main/java/org/apache/hadoop/hbase/util/Bytes.java a53dafe 
  src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java ed8709f 
  src/test/java/org/apache/hadoop/hbase/mapreduce/NMapInputFormat.java PRE-CREATION 
  src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java d04ced2 
  src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java PRE-CREATION

  src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java f1566d3 
  src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java fcb22fb 
  src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java 4595e6e 

Diff: http://review.hbase.org/r/87/diff


Testing
-------

Primary unit/functional testing, a bit of pseudo-distributed testing. Plan on doing full system
tests before commit as well.


Thanks,

Todd


Mime
View raw message