hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pavan Sudheendra <pavan0...@gmail.com>
Subject Re: Is hadoop tread safe?
Date Thu, 29 Aug 2013 14:45:47 GMT
No, I had written a huge Map Reduce program which talks with hbase and does
a lot of computing using it as a source as well as sink.. One of my
colleague saw my code and saw that I had used a lot of static function
instead of making use of proper OOP concepts.. He was telling me that it
shouldn't be the way I should go about doing it.. But my code works fine..
So, was wondering will I face any problem in the future because of this..
That's all.

On Aug 29, 2013 8:11 PM, "Adam Muise" <amuise@hortonworks.com> wrote:

> Mappers don't communicate with each other in traditional MapReduce. If you
> need something more MPI-ish then look to MPI over YARN or write your own
> YARN app.
> If you need multi-threading within the mapper then it is up to you as the
> java developer to make it thread safe. Use the concurrent libraries like
> anything else and Bob's your uncle. Having overly-complicated mappers can
> be difficult to manage however and it kind of misses the mark for MapReduce
> problems.
> Maybe if you expand on your use case a bit someone here can provide
> specific advice.
> On Thu, Aug 29, 2013 at 10:33 AM, Pavan Sudheendra <pavan0591@gmail.com>wrote:
>> Hi all,
>> Is hadoop thread safe? Do mappers make use of threads in any chance? A
>> little bit of information on the way they execute in parallel would help me
>> out. Thanks.
>> Regards,
>> Pavan
> --
> *
> *
> *
> *
> *Adam Muise*
> Solution Engineer
> *Hortonworks*
> amuise@hortonworks.com
> 416-417-4037
> Hortonworks - Develops, Distributes and Supports Enterprise Apache Hadoop.<http://hortonworks.com/>
> Hortonworks Virtual Sandbox <http://hortonworks.com/sandbox>
> Hadoop: Disruptive Possibilities by Jeff Needham<http://hortonworks.com/resources/?did=72&cat=1>
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.

View raw message