hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Muise <amu...@hortonworks.com>
Subject Re: Is hadoop tread safe?
Date Thu, 29 Aug 2013 14:40:58 GMT
Mappers don't communicate with each other in traditional MapReduce. If you
need something more MPI-ish then look to MPI over YARN or write your own
YARN app.

If you need multi-threading within the mapper then it is up to you as the
java developer to make it thread safe. Use the concurrent libraries like
anything else and Bob's your uncle. Having overly-complicated mappers can
be difficult to manage however and it kind of misses the mark for MapReduce

Maybe if you expand on your use case a bit someone here can provide
specific advice.

On Thu, Aug 29, 2013 at 10:33 AM, Pavan Sudheendra <pavan0591@gmail.com>wrote:

> Hi all,
> Is hadoop thread safe? Do mappers make use of threads in any chance? A
> little bit of information on the way they execute in parallel would help me
> out. Thanks.
> Regards,
> Pavan

*Adam Muise*
Solution Engineer

Hortonworks - Develops, Distributes and Supports Enterprise Apache

Hortonworks Virtual Sandbox <http://hortonworks.com/sandbox>

Hadoop: Disruptive Possibilities by Jeff

NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

View raw message