hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Barak Yaish <barak.ya...@gmail.com>
Subject MulitpleOutputs outputs just one line
Date Thu, 24 Jan 2013 07:10:26 GMT

I'm trying to utilize MulitpleOutputs ( hadoop 1.0.4 ) to produce multiple
files based on some policy. In the job i set:

MultipleOutputs.addNamedOutput( job, "rejected", TextOutputFormat.class,
Text.class, NullWritable.class );

And at the mapper:

private MultipleOutputs<Text, Writable> mos;

setup(): mos = new MultipleOutputs( context );

map():   if( somecond )
                     context.write( new Text( key ), NullWritable.get() );
                     logger.info( "Going to write to mos: " + key );
                     mos.write( new Text( key ), NullWritable.get(), "/tmp"

The problem I'm facing is that if multiple mappers running that code, I can
see at the logs that the mos.write() is being invoked, but only one line is
printed to the output file under /tmp. Is there some config I missed?


View raw message