incubator-crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Wills <jwi...@cloudera.com>
Subject Re: getting crunch running on hadoop 2.0.0
Date Wed, 27 Jun 2012 04:31:01 GMT
Oh, and I wanted to mention: getting all of the unit tests to pass
under hadoop 2.0.0 was one of the blockers for getting Crunch added to
bigtop: https://issues.apache.org/jira/browse/BIGTOP-612

Now it's out of the way.

On Tue, Jun 26, 2012 at 9:26 PM, Josh Wills <jwills@apache.org> wrote:
> Hey all,
>
> That last commit from me "fixes" (in some loose sense of the word) the
> WordCountHBaseTest, which was the last test that we needed to get
> working in order to have all of the Crunch unit tests passing under
> Hadoop-2.0.0.
>
> The complexity stems from the fact that the WordCountHBaseTest starts
> up a mini hbase/mapreduce cluster in order to run, and I had to do
> some work to make sure that all of the proper dependencies and class
> definitions that were needed for the test case to pass were available
> to the mini YARN instance that runs w/HBaseTestingUtility under hadoop
> 2.0.0.
>
> Vinod, if you get a chance to take a look at the test case and see if
> there is anything I could have done to make it cleaner, I would be
> much obliged.
>
> J



-- 
Director of Data Science
Cloudera
Twitter: @josh_wills

Mime
View raw message