accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Busbey" <s...@manvsbeard.com>
Subject Re: Review Request 19716: ACCUMULO-2564 Backport changes to unify Hadoop 1/2
Date Thu, 27 Mar 2014 16:31:18 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19716/#review38739
-----------------------------------------------------------


I'm presuming we don't care if unit tests built against one version of Hadoop fail to run
if executed against the libraries of the other?

Open issues:

* src/server/src/main/java/org/apache/accumulo/server/util/CountRowKeys.java uses Counter
directly, issue still exists in 1.5.2-SNAPSHOT
* src/server/src/main/java/org/apache/accumulo/server/test/functional/RunTests.java uses Counter
directly, issue still exists in 1.5.2-SNAPSHOT
* src/examples/simple/src/main/java/org/apache/accumulo/examples/simple/mapreduce/TeraSortIngest.java
uses JobContext directly, issue still exists in 1.5.2-SNAPSHOT, 1.6.0-SNAPSHOT, and master
* src/server/src/main/java/org/apache/accumulo/server/master/LogSort.java makes use of TaskAttemptContext
and JobContext directly, which won't work. I don't know if a fix for Hadoop 1/2 was made before
this class was removed in 1.5.0.
* src/core/src/main/java/org/apache/accumulo/core/client/mapreduce/lib/partition/RangePartitioner.java
uses JobContext directly. I don't know if a fix for Hadoop 1/2 was made before the relevant
methods were changed in a non-API-compatible way.

Additional needed backports:

* 398bc9 ctubbsii ACCUMULO-769 Deprecated and replaced 1.4.x Mapreduce APIs and updated javadocs
for all Mapreduce classes and updated referencing classes.

   In particular, right now you don't include the fix for AccumuloInputFormat, which has a
call to TaskAttemptContext.getConfiguration

* a7e159 ericnewton ACCUMULO-1809 parse short args and use reflection tricks to update counters

   ContinuousMoru uses Counter directly



- Sean Busbey


On March 27, 2014, 6:49 a.m., Mike Drob wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19716/
> -----------------------------------------------------------
> 
> (Updated March 27, 2014, 6:49 a.m.)
> 
> 
> Review request for accumulo, Adam Fuchs, Sean Busbey, Eric Newton, and Josh Elser.
> 
> 
> Bugs: ACCUMULO-2564
>     https://issues.apache.org/jira/browse/ACCUMULO-2564
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> ACCUMULO-2564 Backport changes to unify Hadoop 1/2
> 
> This is a backport of the changes originally made for 1.5.0
> under ACCUMULO-1421 for binary compatability between Hadoop versions
> 1 and 2 with the same Accumulo artifacts. This commit is based on the
> following original work:
> 
> * c9c0d45 (Adam Fuchs)
> * d7ba6ca (Christopher Tubbs)
> * 261cf36 (Eric Newton)
> * cc3c2d8 (Eric Newton)
> 
> 
> Diffs
> -----
> 
>   src/core/src/main/java/org/apache/accumulo/core/client/mapreduce/AccumuloFileOutputFormat.java
7cfab8b6e8f2199620c36a50ac067dee53aab6a9 
>   src/core/src/main/java/org/apache/accumulo/core/client/mapreduce/AccumuloOutputFormat.java
ed0aebf7476d8db6a968e858c9c4c892dac78bc5 
>   src/core/src/main/java/org/apache/accumulo/core/client/mapreduce/InputFormatBase.java
a11096c3f855104db4e3b78a49f3fb039ac268ec 
>   src/server/src/main/java/org/apache/accumulo/server/Accumulo.java 184692c48ce8013067053c1d0f0dd6a7a889473a

>   src/server/src/main/java/org/apache/accumulo/server/test/continuous/ContinuousVerify.java
9441cf5314383705260d6fa85e69597525de421c 
> 
> Diff: https://reviews.apache.org/r/19716/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Mike Drob
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message