hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@hortonworks.com>
Subject Re: Update on hadoop-0.23
Date Wed, 28 Sep 2011 20:55:19 GMT
Roman,

On Sep 27, 2011, at 12:24 PM, Roman Shaposhnik wrote:

> I'm sorry if I had too many questions in my original email, but I can't find
> an answer to my "integration tests" question. Could you, please, share
> a URL with us where I can find out more about them?

As you know, me & my team at HW along with folks at Y do a lot of manual testing along
with tests like GridMix/PigMix etc.

The basic idea is to test all features of HDFS, MapReduce, Streaming, Pipes etc. Similarly
for Pig, Hive, Oozie. I'm sure none of this is news to you.

Similarly we test for performance for all aspects of MapReduce.

> On Mon, Sep 26, 2011 at 11:20 PM, Arun C Murthy <acm@hortonworks.com> wrote:
>> # We made changes to Pig - rather we got help from the Pig team, particularly Daniel.
>> 
>> So, we plan to work through the rest of the stack - Hive, Oozie etc. very soon and
we'll
>> depend on updated releases from the individual projects.
> 
> Do we have any kinds of commitment from downstream projects as far as those
> updates are concerned?

The easiest way to get 'commitment' is to test and provide patches as necessary:
# mapreduce-dev@ helped pig-dev@ to do it for Pig
# Todd has done it for HBase, hdfs-dev@ will further help as necessary
# mapreduce-dev@ will soon help out for Hive, Oozie 

... etc.

> Are they targeting these changes as part of point (patch)
> release of an already released version (like Pig 0.9.X for example) or
> will it be
> part of a brand new major release?


I don't know about specific releases for all projects: 
# Pig will work 0.9.1 or 0.9.2 and beyond.
# HBase trunk works (refer to Todd's msg) - the actual release depends on HBase community
for a release.

Historically, we make a release of Hadoop Core and then work through related projects to make
necessary changes - mostly minor, sometime more. Thus, having an early release is important.

Arun


Mime
View raw message