hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2520) InputSampler.RandomSampler only accepts Text keys
Date Thu, 19 May 2011 20:55:47 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13036450#comment-13036450
] 

Chris Douglas commented on MAPREDUCE-2520:
------------------------------------------

Which version of Hadoop are you using? I don't see this error in [0.20|http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20/src/mapred/org/apache/hadoop/mapred/lib/InputSampler.java],
[0.21|http://svn.apache.org/repos/asf/hadoop/mapreduce/branches/branch-0.21/src/java/org/apache/hadoop/mapreduce/lib/partition/InputSampler.java],
or [trunk|http://svn.apache.org/repos/asf/hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapreduce/lib/partition/InputSampler.java].

> InputSampler.RandomSampler only accepts Text keys
> -------------------------------------------------
>
>                 Key: MAPREDUCE-2520
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2520
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: William McNeill
>            Priority: Minor
>
> I want to do a total sort on some data whose key type is Writable but not Text.  I wrote
an InputSampler.RandomSampler object following the example in the "Total Sort" section of
Hadoop: The Definitive Guide.  When I call InputSampler.writePartitionFile() I get a runtime
class cast exception because my key type cannot be cast to Text.  Specifically the issue seems
to be the following section of InputSampler.getSample():
>     K key = reader.getCurrentKey();
>     ....
>     Text keyCopy = WritableUtils.<Text>clone((Text)key, job.getConfiguration());
> You can only use a RandomSampler on data with Text keys despite the fact that InputSampler
takes <Key, Value> generic parameters.
> InputSampler.getSample() should be changed to cast the key to type K instead of type
Text.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message