hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Tanquary <matt.tanqu...@gmail.com>
Subject Cannot find MultipleOutputs files
Date Mon, 22 Nov 2010 16:43:02 GMT
I am trying to implement MultipleOutputs (the new API). I have done the
following:

Added the following to my Tool run() method:

MultipleOutputs.addNamedOutput(sortJob, "l1n1", TextOutputFormat.class,
NullWritable.class, Text.class);
MultipleOutputs.addNamedOutput(sortJob, "l1n2", TextOutputFormat.class,
NullWritable.class, Text.class);


Here's my reducer:

static class FinalSortReducer extends Reducer<TextTriplet, IntWritable,
NullWritable, Text>
    {
        private MultipleOutputs<NullWritable, Text> mos;

        public void setup(Context context) {
          mos = new MultipleOutputs<NullWritable, Text>(context);
       }
        public void reduce(TextTriplet key, Iterable<IntWritable> values,
                Context context) throws IOException, InterruptedException {

            Text out = new Text();

            for (IntWritable val : values) {
                context.getCounter(TotalCounter.ITEM).increment(1);
                out.set(key.getThird() + "\t" + key.getSecond());
                if (key.getFirst().toString().equals("l1n1"))
                {
                    //mos.write(NullWritable.get(), out,
key.getFirst().toString());
                    mos.write("l1n1", NullWritable.get(), key.getThird() +
"\t" + key.getSecond());
                    //logger.info("Wrote: " + key.toString());
                }else if (key.getFirst().toString().equals("l1n2"))
                {
                    //mos.write(NullWritable.get(), out,
key.getFirst().toString());
                    mos.write("l1n2", NullWritable.get(), key.getThird() +
"\t" + key.getSecond());
                    //logger.info("Wrote: " + key.toString());
                } else
                {
                    context.write(NullWritable.get(), out);
                }
            }
        }

        public void cleanup(Context context) throws IOException,
InterruptedException {
            mos.close();
        }
    }

Once the job is finished, I end up with nothing in my output path although
statistics show that my reducer output 83 records.
The only thing found in my output path is '_logs' folder.

I have searched up and down the folder tree to see if maybe the outputs are
somewhere else, but have not been able to find the output.

Any idea about what I might be doing wrong?

Thanks!
-M@

-- 
Have you thanked a teacher today? ---> http://www.liftateacher.org

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message