hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hemanth Yamijala <yhema...@gmail.com>
Subject Re: Can we write output directly to HDFS from Mapper
Date Fri, 28 Sep 2012 04:44:37 GMT
Can certainly do that. Indeed, if you set the number of reducers to 0,
the map output will be directly written to HDFS by the framework
itself. You may also want to look at
http://hadoop.apache.org/docs/stable/mapred_tutorial.html#Task+Side-Effect+Files
to see some things that need to be taken care of if you are writing
files on your own.

Thanks
hemanth

On Fri, Sep 28, 2012 at 9:45 AM, Balaraman, Anand
<Anand_Balaraman@syntelinc.com> wrote:
> Hi
>
>
>
> In Map-Reduce, is it appropriate to write the output directly to HDFS from
> Mapper (without using a reducer) ?
>
> Are there any adverse effects in doing so or are there any best practices to
> be followed in this aspect ?
>
>
>
> Comments are much appreciable at the moment J
>
>
>
> Thanks and Regards
>
> Anand B
>
> Confidential: This electronic message and all contents contain information
> from Syntel, Inc. which may be privileged, confidential or otherwise
> protected from disclosure. The information is intended to be for the
> addressee only. If you are not the addressee, any disclosure, copy,
> distribution or use of the contents of this message is prohibited. If you
> have received this electronic message in error, please notify the sender
> immediately and destroy the original message and all copies.

Mime
View raw message