hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jamal sasha <jamalsha...@gmail.com>
Subject Writing to multiple directories in hadoop
Date Fri, 11 Oct 2013 22:19:21 GMT
Hi,

I am trying to separate my output from reducer to different folders..

My dirver has the following code:
 FileOutputFormat.setOutputPath(job, new Path(output));
            //MultipleOutputs.addNamedOutput(job, namedOutput,
outputFormatClass, keyClass, valueClass)
            //MultipleOutputs.addNamedOutput(job, namedOutput,
outputFormatClass, keyClass, valueClass)
            MultipleOutputs.addNamedOutput(job, "foo",
TextOutputFormat.class, NullWritable.class, Text.class);
            MultipleOutputs.addNamedOutput(job, "bar",
TextOutputFormat.class, Text.class,NullWritable.class);
            MultipleOutputs.addNamedOutput(job, "foobar",
TextOutputFormat.class, Text.class, NullWritable.class);

And then my reducer has the following code:
mos.write("foo",NullWritable.get(),new Text(jsn.toString()));
mos.write("bar", key,NullWritable.get());
mos.write("foobar", key,NullWritable.get());

But in the output, I see:

output/foo-r-0001
output/foo-r-0002
output/foobar-r-0001
output/bar-r-0001


But what I am trying is :

output/foo/part-r-0001
output/foo/part-r-0002
output/bar/part-r-0001
output/foobar/part-r-0001

How do I do this?
Thanks

Mime
View raw message