incubator-bigtop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roman Shaposhnik <...@apache.org>
Subject Deciding on the Hadoop version for Bigtop 0.2.0
Date Fri, 30 Sep 2011 19:06:15 GMT
Hi!

By now it is obvious that the decision on what version of Hadoop to use
for Bigtop 0.2.0 is not going to be an easy one ;-)

So far we've got the following choices:
   1. Do nothing and stick with 0.20.2. This is not all that bad, I suppose,
       we've got tons of updates to the Bigtop to justify a release. But it
       feels a little bit anticlimactic in a sense that Bigtop 0.3.0 is very
       likely to be on top of 0.23 and I really would like to have a chance
       of having a Bigtop release on top of the last MR1.
   2. Target upcoming Hadoop 0.22. This is going to break all of the
       downstream except for HBase. Here's a list of JIRAs filed:
           https://issues.apache.org/jira/browse/SQOOP-354
           https://issues.apache.org/jira/browse/PIG-2277
           https://issues.apache.org/jira/browse/OOZIE-565
           https://issues.apache.org/jira/browse/HIVE-2468
           https://issues.apache.org/jira/browse/MAHOUT-822
        As you can see, most of the downstream is OKins with
        implementing the changes in time for .23, but not necessarily
        in time for .22 and Bigtop 0.2.0.
   3. Target 0.20.205.0. The downside there is that unless the following
        build issues are resolved, we can't even compile it on the platforms
        Bigtop cares about:
             https://issues.apache.org/jira/browse/HADOOP-6436
             https://issues.apache.org/jira/browse/MAPREDUCE-2127
             https://issues.apache.org/jira/browse/HDFS-2327

For #2 and #3 we can, potentially, mitigate the timing issue by patching
the downstream locally at the Bigtop level. Do we want to entertain such
an idea?

Anyway, please chime in with your thoughts. It would be very nice to
have Bigtop 0.2.0 fully tested and out sometime around first weeks of
Nov. As such we have to make a decision on Hadoop rather soon.

Thanks,
Roman.

Mime
View raw message