hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@hortonworks.com>
Subject Re: Update on hadoop-0.23
Date Tue, 27 Sep 2011 06:20:41 GMT

In general, we'll need to make changes upstream: 
# I believe someone got HBase working. 
# We made changes to Pig - rather we got help from the Pig team, particularly Daniel.

So, we plan to work through the rest of the stack - Hive, Oozie etc. very soon and we'll depend
on updated releases from the individual projects.


On Sep 26, 2011, at 3:15 PM, Roman Shaposhnik wrote:

> Hi Arun!
> Great news! Hopefuly you wouldn't mind answering some of the questions below...
> On Mon, Sep 26, 2011 at 2:07 PM, Arun C Murthy <acm@hortonworks.com> wrote:
>> NextGen MapReduce (aka MRv2, aka YARN) is coming along great:
>> # We are happy to report we've done extensive scale testing to confirm stability
>>  - Sort/GridMixv3 etc. at ~350nodes
>>  - Scale testing with simulated clusters of ~1500 nodes
>> # Functional tests for all of MapReduce functionality
>> # Pig  (0.9 & 0.9.1) working with NextGen MapReduce
> Is there a *released* version of Pig that compiles cleanly against .23
> snapshots?
> Same question for Hive.
>> We are about to finish performance certification for both HDFS & MapReduce in
the next
>> couple of weeks too, after which we start integration tests with HBase, Hive, Oozie
> I'm curious -- what are these integrations tests? Can I take a look at
> them? I would
> be really nice if we can levarage those via Bigtop infrastructure. Currently we
> have a certain # of integration tests in Bigtop that we're running
> against a fully
> deployed stack, but it would be quite nice to have extra coverage.
>> Given where we are I'm confident we can have a strong hadoop-0.23.0 release
>> by late October. The current plan is to deploy to alpha clusters in November. Citius,
Altius, Fortius! :)
> Could you, please, elaborate on what will be part of that deployment?
> Which versions
> of Pig, Hive, HBase, Oozie and Mahout are you targeting?
> Thanks,
> Roman.

View raw message