crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CRUNCH-556) Fix total sorts in Crunch-on-Spark
Date Thu, 13 Aug 2015 05:15:45 GMT
Josh Wills created CRUNCH-556:
---------------------------------

             Summary: Fix total sorts in Crunch-on-Spark
                 Key: CRUNCH-556
                 URL: https://issues.apache.org/jira/browse/CRUNCH-556
             Project: Crunch
          Issue Type: Bug
          Components: Spark
    Affects Versions: 0.13.0
            Reporter: Josh Wills
             Fix For: 0.14.0


>From the user mailing list, trying to perform a total sort to create an HFile w/Crunch
on Spark throws the following exception: https://gist.github.com/anonymous/920c000f20229eaa76d8

The problem can be traced to not properly configuring the partitioner w/the path to the partition
file that is stored in the GroupingOptions extra configuration settings. These settings get
passed correctly for the MR job, but not for the Spark ones.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message