hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erik Test <erik.shi...@gmail.com>
Subject Set variables in mapper
Date Mon, 02 Aug 2010 16:17:03 GMT
Hi,

I'm trying to set a variable in my mapper class by reading an argument from
the command line and then passing the entry to the mapper from main. Is this
possible?

  public static void main(String[] args) throws Exception
  {
    JobConf conf = new JobConf(DistanceCalc2.class);
    conf.setJobName("Calculate Distances");

    conf.setOutputKeyClass(Text.class);
    conf.setOutputValueClass(DoubleWritable.class);

    conf.setMapperClass(Map.class);
    //conf.setReducerClass(Reduce.class);

    conf.setInputFormat(TextInputFormat.class);
    conf.setOutputFormat(TextOutputFormat.class);

    FileInputFormat.setInputPaths(conf, new Path(args[0]));
    FileOutputFormat.setOutputPath(conf, new Path(args[1]));

    Map.setN(args[2]);

    JobClient.runJob(conf);
  }//main


  public static class Map extends MapReduceBase
    implements Mapper<LongWritable, Text,
      Text, DoubleWritable>
        {
               ...
               private static int N;

               ...

               public void map(LongWritable key, Text value,
                 OutputCollector<Text, DoubleWritable> output,
                  Reporter reporter) throws IOException
                {
                    ....
                    dim = tokens.length / N;
                    ...
                }

               public static void setN(String newN)
               {
                  N = Integer.parseInt(newN);
               }
        }

I've tried the code above but I get an error saying that I'm dividing by
zero. Obviously, the argument I enter for N isn't being set as specified.
Erik

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message