incubator-crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias Friedrich (JIRA)" <>
Subject [jira] [Commented] (CRUNCH-87) crunch-archtype fails with crunch.platform=2
Date Fri, 05 Oct 2012 15:36:02 GMT


Matthias Friedrich commented on CRUNCH-87:

Hmm. It would be no problem to depend on hadoop-core for Hadoop 1 and hadoop-client for Hadoop
2, that would make it compile again. The problem is a bit more serious and it goes back to
CRUNCH-4. Even if a client project depends on hadoop-client-2.x it will still get our dependencies
for Hadoop 1. There is no way in Maven for a client project to activate a profile via its
POM properties.

As a consequence, no Crunch-based Hadoop-2 project will work locally in the IDE unless the
user adds correct dependencies himself (at least commons-cli, commons-httpclient, avro, and
a compatible version of slf4j-api). I see no way of fixing this other than fixing Hadoop.
Ideas, anyone?
> crunch-archtype fails with crunch.platform=2
> --------------------------------------------
>                 Key: CRUNCH-87
>                 URL:
>             Project: Crunch
>          Issue Type: Bug
>    Affects Versions: 0.4.0
>            Reporter: Josh Wills
>            Assignee: Matthias Friedrich
> Running:
> mvn clean install -Dcrunch.platform=2
> fails on the crunch-archetype submodule b/c the hadoop-core dependency doesn't exist
for Hadoop 2.0.0 (and I don't believe it's coming back.) Can we explicitly add hadoop-client
and the other dependencies the IDE would need to run the LocalJobRunner?

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message