hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Patrick Kling" <pkl...@cs.uwaterloo.ca>
Subject Re: Review Request: Refactor RaidNode to remove dependency on map reduce
Date Wed, 03 Nov 2010 02:26:47 GMT


> On 2010-11-02 16:02:02, Ramkumar Vadali wrote:
> > Please post ant test and ant test-patch results.
> 
> Patrick Kling wrote:
>     ant test in src/contrib/raid:
>     
>         [junit] Running org.apache.hadoop.hdfs.TestRaidDfs
>         [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 60.289 sec
>         [junit] Running org.apache.hadoop.raid.TestBlockFixer
>         [junit] Tests run: 5, Failures: 0, Errors: 1, Time elapsed: 141.91 sec
>         [junit] Running org.apache.hadoop.raid.TestBlockFixer
>         [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 125.852 sec
>         [junit] Running org.apache.hadoop.raid.TestDirectoryTraversal
>         [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 9.951 sec
>         [junit] Running org.apache.hadoop.raid.TestErasureCodes
>         [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 23.623 sec
>         [junit] Running org.apache.hadoop.raid.TestGaloisField
>         [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 0.27 sec
>         [junit] Running org.apache.hadoop.raid.TestHarIndexParser
>         [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.024 sec
>         [junit] Running org.apache.hadoop.raid.TestRaidFilter
>         [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 7.017 sec
>         [junit] Running org.apache.hadoop.raid.TestRaidHar
>         [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 68.747 sec
>         [junit] Running org.apache.hadoop.raid.TestRaidNode
>         [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 462.504 sec
>         [junit] Running org.apache.hadoop.raid.TestRaidPurge
>         [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 119.206 sec
>         [junit] Running org.apache.hadoop.raid.TestRaidShell
>         [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 22.446 sec
>     
>     test:
>     
>     ant test-patch:
>     
>     BUILD SUCCESSFUL
>     
>          [exec] +1 overall.  
>          [exec] 
>          [exec]     +1 @author.  The patch does not contain any @author tags.
>          [exec] 
>          [exec]     +1 tests included.  The patch appears to include 18 new or modified
tests.
>          [exec] 
>          [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
>          [exec] 
>          [exec]     +1 javac.  The applied patch does not increase the total number of
javac compiler warnings.
>          [exec] 
>          [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
>          [exec] 
>          [exec]     +1 release audit.  The applied patch does not increase the total
number of release audit warnings.
>          [exec] 
>          [exec]     +1 system test framework.  The patch passed system test framework
compile.
>          [exec] 
>          [exec] 
>          [exec] 
>          [exec] 
>          [exec] ======================================================================
>          [exec] ======================================================================
>          [exec]     Finished build.
>          [exec] ======================================================================
>          [exec] ======================================================================
>          [exec] 
>          [exec] 
>     
>     BUILD SUCCESSFUL
>     
>

Note that the test-patch results were obtained after updating test-patch.properties to reflect
the 25 findbugs warning and 3 release audit warnings that I get on a clean trunk (see MAPREDUCE-2172
for details).


- Patrick


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24/#review24
-----------------------------------------------------------


On 2010-11-02 15:59:44, Patrick Kling wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24/
> -----------------------------------------------------------
> 
> (Updated 2010-11-02 15:59:44)
> 
> 
> Review request for hadoop-mapreduce.
> 
> 
> Summary
> -------
> 
> Refactor RaidNode to remove dependency on map reduce
> 
> I am refactoring the RaidNode code as follows: The base class RaidNode will contain the
common functionality needed for raiding files. The derived class LocalRaidNode contains an
implementation of RaidNode that performs raiding locally. The derived class DistRaidNode performs
raiding using map reduce jobs. This way, only DistRaidNode has a dependency on map reduce
code and RaidNode and LocalRaidNode can be moved to HDFS.
> 
> 
> This addresses bug MAPREDUCE-2142.
>     https://issues.apache.org/jira/browse/MAPREDUCE-2142
> 
> 
> Diffs
> -----
> 
>   http://svn.apache.org/repos/asf/hadoop/mapreduce/trunk/src/contrib/raid/README 1030210

>   http://svn.apache.org/repos/asf/hadoop/mapreduce/trunk/src/contrib/raid/src/java/org/apache/hadoop/raid/DistRaidNode.java
PRE-CREATION 
>   http://svn.apache.org/repos/asf/hadoop/mapreduce/trunk/src/contrib/raid/src/java/org/apache/hadoop/raid/Encoder.java
1030210 
>   http://svn.apache.org/repos/asf/hadoop/mapreduce/trunk/src/contrib/raid/src/java/org/apache/hadoop/raid/LocalRaidNode.java
PRE-CREATION 
>   http://svn.apache.org/repos/asf/hadoop/mapreduce/trunk/src/contrib/raid/src/java/org/apache/hadoop/raid/RaidNode.java
1030210 
>   http://svn.apache.org/repos/asf/hadoop/mapreduce/trunk/src/contrib/raid/src/java/org/apache/hadoop/raid/RaidUtils.java
1030210 
>   http://svn.apache.org/repos/asf/hadoop/mapreduce/trunk/src/contrib/raid/src/test/org/apache/hadoop/hdfs/TestRaidDfs.java
1030210 
>   http://svn.apache.org/repos/asf/hadoop/mapreduce/trunk/src/contrib/raid/src/test/org/apache/hadoop/raid/TestBlockFixer.java
1030210 
>   http://svn.apache.org/repos/asf/hadoop/mapreduce/trunk/src/contrib/raid/src/test/org/apache/hadoop/raid/TestRaidHar.java
1030210 
>   http://svn.apache.org/repos/asf/hadoop/mapreduce/trunk/src/contrib/raid/src/test/org/apache/hadoop/raid/TestRaidNode.java
1030210 
>   http://svn.apache.org/repos/asf/hadoop/mapreduce/trunk/src/contrib/raid/src/test/org/apache/hadoop/raid/TestRaidPurge.java
1030210 
>   http://svn.apache.org/repos/asf/hadoop/mapreduce/trunk/src/contrib/raid/src/test/org/apache/hadoop/raid/TestRaidShell.java
1030210 
> 
> Diff: https://reviews.apache.org/r/24/diff
> 
> 
> Testing
> -------
> 
> Raid unit tests
> 
> 
> Thanks,
> 
> Patrick
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message