hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: How to speed up Hadoop?
Date Fri, 06 Sep 2013 02:38:23 GMT
I'd recommend reading Eric Sammer's "Hadoop Operations" (O'Reilly)
book. It goes over a lot of this stuff - building, monitoring, tuning,
optimizing, etc..

If your goal is just speed and quicker results, and not retention or
safety, by all means use replication factor as 1. Note that its
difficult for us to suggest configs unless you also share your
use-case (in brief) or goals. While the software is highly tunable, a
lot of tweaks depend on what you are planning to do.

On Fri, Sep 6, 2013 at 6:11 AM, Sundeep Kambhampati
<kambhamp@cse.ohio-state.edu> wrote:
> Hi all,
>     I am looking for ways to configure Hadoop inorder to speed up data
> processing. Assuming all my nodes are highly fault tolerant, will making
> data replication factor 1 speed up the processing? Are there some way to
> disable failure monitoring done by Hadoop?
> Thank you for your time.
> -Sundeep

Harsh J

View raw message