hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shrijeet Paliwal <shrij...@rocketfuel.com>
Subject Re: Passing configuration options to the Map class
Date Mon, 13 Dec 2010 21:27:50 GMT
Hi,

configure(JobConf conf)method in
http://hadoop.apache.org/common/docs/r0.20.0/api/org/apache/hadoop/mapred/MapReduceBase.html
is your friend.

Try something like this,


public class BobsMapper extends MapReduceBase implements
Mapper<LongWritable, Text, Text, Text> {
    private Path out_path = null;
    /**
     * Use to configure map job.
     * Parameters from configure XML can be accessed here.
     * This will be called once before framework start calling map()
iteratively for given input file
     */
    public void configure(JobConf conf) {
        try {
            super.configure(conf);
            //get parameter values from configuration XML
            out_path = new Path(conf.get("outputPath"));
        }
        catch (Exception e) {
            _log.error("Problem configuring map job. " + e.getMessage());
        }
    }

    /** the mapper should not be able to access out_path.
    public void map(...) {
    }
}

On Mon, Dec 13, 2010 at 12:54 PM, Eric <eric.xkcd@gmail.com> wrote:
>
> Hi,
>
> I'm looking for a way to pass options to my Map class. More specifically,
I want my mappers to write to sequence files in a path that I can specify on
the command line. I tried setting a variable with:
>
> Jobconf conf = ........;
> conf.set("outputPath", "/foo/bar");
>
> (I'm using the deprecated libraries)
>
> I can not retrieve this setting in my mapper however. Is there another
way, or am I doing something wrong?
>
> Best regards,
> Eric

Mime
View raw message