incubator-crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <>
Subject [jira] [Commented] (CRUNCH-4) Create profiles for Crunch builds based on Hadoop 1.0 and Hadoop 2.0
Date Thu, 09 Aug 2012 22:47:19 GMT


Josh Wills commented on CRUNCH-4:

So I downloaded the 0.94.1 HBase release and verified that Crunch patches tests with it--
*IF* you download the HBase source files and compile it yourself using:

mvn clean install -Dhadoop.profile=2.0 -DskipTests

and then run crunch as:

mvn clean verify -Dcrunch.platform=2

The default maven jars go against the 1.0.x version of Hadoop and will fail if you run them
with Hadoop 2.0.0-alpha. I'm not sure of the best way to message that to people short of putting
a note on the website, at least until the HBase guys release a maven version that is compiled
against 2.0.0-alpha. In any case, I'd like to integrate Rahul's changes so that we can commit
this, break the HBase stuff into its own subcomponent, finish CRUNCH-16, and release.

> Create profiles for Crunch builds based on Hadoop 1.0 and Hadoop 2.0
> --------------------------------------------------------------------
>                 Key: CRUNCH-4
>                 URL:
>             Project: Crunch
>          Issue Type: Task
>          Components: Core
>            Reporter: Josh Wills
>            Assignee: Rahul Sharma
>              Labels: maven, newbie
>         Attachments: 0001-CRUNCH-4-Profiles-for-hadoop-0.20-and-2.0-versions.patch, 0001-Created-profiles-for-CDH3-hbase-.90.6-and-hadoop2-hb.patch,
CRUNCH-4-Hadoop-Profiles.patch, CRUNCH-4-No-Vendor-Specific-Profiles.patch, CRUNCH-4-Profiles-for-hadoop-verions.patch
> We need to create two Maven build profiles for Crunch, one that depends on Hadoop 1.0
(probably 1.0.3 for Hadoop and 0.90.5 for HBase) and Hadoop 2.0 (2.0.0-alpha for Hadoop and
a stable 0.92.x for HBase.)
> Reference:

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message