crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <>
Subject [jira] [Commented] (CRUNCH-518) Can't build crunch-spark due to protobuf dependency
Date Fri, 08 May 2015 21:25:02 GMT


Josh Wills commented on CRUNCH-518:

Ack, sorry-- meant CRUNCH-517, which I had meant to commit but forgot to in all of the fun
at Strata the past few days.

I'd like to get this in, along with CRUNCH-509 and CRUNCH-517, and then try to kick off a
release of 0.12. After that gets out, I'm really looking forward to an omnibus patch that
rips out all of the hadoop1 stuff, upgrades the HBase dependencies to 1.0, and gets us back
to a single build.

> Can't build crunch-spark due to protobuf dependency
> ---------------------------------------------------
>                 Key: CRUNCH-518
>                 URL:
>             Project: Crunch
>          Issue Type: Bug
>            Reporter: Gabriel Reid
>         Attachments: CRUNCH-518.patch
> When trying to build clean build, I get errors like the following from crunch-spark:
> {code}
> [ERROR] /Users/greid/development/apache-projects/crunch/crunch-spark/src/it/scala/org/apache/crunch/scrunch/spark/PageRankTest.scala:49:
error: bad symbolic reference. A signature in PTypeH.class refers to term protobuf
> [ERROR] in package which is not available.
> [ERROR] It may be completely missing from the current classpath, or the version on
> [ERROR] the classpath might be incompatible with the version used when compiling PTypeH.class.
> [ERROR]     prev.cogroup(out).map((url, v) => {
> {code}
> It seems that this is due to protobuf being referenced from PTypeH, but since it's in
provided scope in crunch-scrunch, it doesn't get pulled in when building crunch-spark.

This message was sent by Atlassian JIRA

View raw message