crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gabriel Reid (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CRUNCH-96) Add secondary sort functionality to o.a.c.lib
Date Wed, 17 Oct 2012 18:46:04 GMT

    [ https://issues.apache.org/jira/browse/CRUNCH-96?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478223#comment-13478223
] 

Gabriel Reid commented on CRUNCH-96:
------------------------------------

Just took a look through this. Out of curiosity, is there that much of a use case specifically
for this? I'm definitely not against adding it, but I'm also wondering if this is a really
specific use case that we need to support, or if we can make it more generic somehow.

I think that a call to GroupingOptions.Builder#numReducers is needed to set the number of
reducers to 1 until CRUNCH-51 is taken care of (that's kind of on the backburner at the moment,
but I am still working on it).

More info in the javadocs about what the class does would be good. The SecondarySort example
explains it pretty well, but it's not directly clear how to use SecondarySort when you look
at the library class.

It might also be good to change the class name of the SecondarySort example (as it stands,
there are two SecondarySort classes). Same goes for the input files for secondary_sort_input.txt
file.






                
> Add secondary sort functionality to o.a.c.lib
> ---------------------------------------------
>
>                 Key: CRUNCH-96
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-96
>             Project: Crunch
>          Issue Type: Improvement
>          Components: Core, MapReduce Patterns
>            Reporter: Josh Wills
>            Assignee: Josh Wills
>             Fix For: 0.4.0
>
>         Attachments: CRUNCH-96.patch
>
>
> I've been working on a problem that required a secondary sorting pattern that was very
similar to the example that Alex Kozlov created in CRUNCH-78, so it would be good to extract
the pattern from the example and move it to o.a.c.lib so it can be easily available to clients.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message