hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan Rawson" <ryano...@gmail.com>
Subject Re: Review Request: HBASE-1923. Bulk load into existing tables
Date Wed, 26 May 2010 06:51:57 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/87/#review76
-----------------------------------------------------------



src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java
<http://review.hbase.org/r/87/#comment396>

    this cant go here, HFile cannot know about KeyValues. It will go into StoreFile
    



src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java
<http://review.hbase.org/r/87/#comment397>

    you might also say the last key in the HFile means the last "KeyValue"



src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
<http://review.hbase.org/r/87/#comment398>

    this won't work for an existing table - if the current time millis > the current sequence
id in the live server, we could end up losing edits during a log recovery


- Ryan


On 2010-05-25 14:21:16, Todd Lipcon wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://review.hbase.org/r/87/
> -----------------------------------------------------------
> 
> (Updated 2010-05-25 14:21:16)
> 
> 
> Review request for hbase, stack and Jonathan Gray.
> 
> 
> Summary
> -------
> 
> Here's a first patch that implements bulk import into existing tables. This applies on
top of HBASE-2586 and HBASE-2588 - I've pushed the series of the three to my github: http://github.com/toddlipcon/hbase/tree/hfof-review
> 
> I have some TODOs left that I want to take care of before this gets committed, but since
it's a pretty large patch, I figured I'd get it out for review ASAP.
> 
> The stuff in the hadoopbackport package is essentially copypaste from Hadoop trunk, so
you can ignore that in the review.
> 
> 
> This addresses bug HBASE-1923.
>     http://issues.apache.org/jira/browse/HBASE-1923
> 
> 
> Diffs
> -----
> 
>   pom.xml 0a009cf 
>   src/main/java/org/apache/hadoop/hbase/HRegionInfo.java 29b0cd6 
>   src/main/java/org/apache/hadoop/hbase/io/ImmutableBytesWritable.java 0a9ec4b 
>   src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java cf4768f 
>   src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 4cbe52a 
>   src/main/java/org/apache/hadoop/hbase/mapreduce/Driver.java 3d40695 
>   src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java 9c8e53e 
>   src/main/java/org/apache/hadoop/hbase/mapreduce/ImportTsv.java PRE-CREATION 
>   src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java PRE-CREATION

>   src/main/java/org/apache/hadoop/hbase/mapreduce/PutSortReducer.java PRE-CREATION 
>   src/main/java/org/apache/hadoop/hbase/mapreduce/SimpleTotalOrderPartitioner.java af3d588

>   src/main/java/org/apache/hadoop/hbase/mapreduce/hadoopbackport/InputSampler.java PRE-CREATION

>   src/main/java/org/apache/hadoop/hbase/mapreduce/hadoopbackport/TotalOrderPartitioner.java
PRE-CREATION 
>   src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 287cd48 
>   src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java fb65ed1 
>   src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 7de766d 
>   src/main/java/org/apache/hadoop/hbase/util/Bytes.java a53dafe 
>   src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java ed8709f 
>   src/test/java/org/apache/hadoop/hbase/mapreduce/NMapInputFormat.java PRE-CREATION 
>   src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java d04ced2

>   src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java PRE-CREATION

>   src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java fcb22fb 
> 
> Diff: http://review.hbase.org/r/87/diff
> 
> 
> Testing
> -------
> 
> Primary unit/functional testing, a bit of pseudo-distributed testing. Plan on doing full
system tests before commit as well.
> 
> 
> Thanks,
> 
> Todd
> 
>


Mime
View raw message