hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4143) Support for a "raw" Partitioner that partitions based on the serialized key and not record objects
Date Wed, 10 Sep 2008 18:15:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12629895#action_12629895
] 

Doug Cutting commented on HADOOP-4143:
--------------------------------------

> Unfortunately, Partitioner is an interface [ ... ] 

This is a non-issue with HADOOP-1230.  If this feature is needed soon, then we should push
harder on that one.

That said, I'm still not clear on the motivation.  Is it performance?  Comparators already
provide both raw and cooked comparisons.  If a partitioner is defined in terms of a comparator,
it must currently used a cooked comparison, which might be slower.  If this is a performance
issue, then we should measure the potential performance improvement with a benchmark before
we consider the API change.  Are there non-performance reasons for this change?

> Support for a "raw" Partitioner that partitions based on the serialized key and not record
objects
> --------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-4143
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4143
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Chris Douglas
>         Attachments: 4143-0.patch
>
>
> For some partitioners (particularly those using comparators to classify keys), it would
be helpful if one could specify a "raw" partitioner that would receive the serialized version
of the key rather than the object emitted from the map.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message