hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Young Kim <juneng...@gmail.com>
Subject MultipleOutputs is not working on 0.20.2 properly.
Date Thu, 20 Jan 2011 09:32:08 GMT
Hi,

I am using Hadoop 0.20.2 version on my cluster.

To write multiple output files from a reducer, I want to use 
MultipleOutputs class.

in this class, I need to call addNamedOutput.


      addNamedOutput

public static void*addNamedOutput*(JobConf  <http://hadoop.apache.org/common/docs/r0.20.2/api/org/apache/hadoop/mapred/JobConf.html>
 conf,
                                   String  <http://java.sun.com/javase/6/docs/api/java/lang/String.html?is-external=true>
 namedOutput,
                                   Class  <http://java.sun.com/javase/6/docs/api/java/lang/Class.html?is-external=true><?
extendsOutputFormat  <http://hadoop.apache.org/common/docs/r0.20.2/api/org/apache/hadoop/mapred/OutputFormat.html>>
 outputFormatClass,
                                   Class  <http://java.sun.com/javase/6/docs/api/java/lang/Class.html?is-external=true><?>
 keyClass,
                                   Class  <http://java.sun.com/javase/6/docs/api/java/lang/Class.html?is-external=true><?>
 valueClass)

    Adds a named output for the job.

    *Parameters:*
        |conf|- job conf to add the named output
        |namedOutput|- named output name, it has to be a word, letters
        and numbers only, cannot be the word 'part' as that is reserved
        for the default output.
        |outputFormatClass|- OutputFormat class.
        |keyClass|- key class
        |valueClass|- value class


As you see, this method takes JobConf type as a first argument.
but, this one is deprecated one in 0.20.2.

additionally, MultipleOuputs class is only stored in 
org.apache.hadoop.mapred.lib.MultipleOutputs.
(not in org.apache.hadoop.mapred*uce*.lib.MultipleOutputs)

this is related discussions about this problem.
https://issues.apache.org/jira/browse/HADOOP-3149
https://issues.apache.org/jira/browse/MAPREDUCE-370


How I can set multiple output on my version?
thanks.

-- 

-----
Junyoung Kim (juneng603@gmail.com)


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message