hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Derek Gottfrid" <de...@codecubed.com>
Subject Re: configuration suggestions for 1k nodes
Date Wed, 07 Nov 2007 11:38:52 GMT
excellent. I will keep the group posted as I progress.

On 11/7/07, Raghu Angadi <rangadi@yahoo-inc.com> wrote:
>
> Not a complete list by far, but just a start :
>
> For HDFS:
>
> - Make sure you run Java 6 (jdk1.6).
> - Set namenode handler count to 40 or more (dfs.namenode.handler.count,
> and may be  mapred.job.tracker.handler.count etc).
>
> - more config guides are in the works :
> https://issues.apache.org/jira/browse/HADOOP-1917
>
> Raghu.
>
> Derek Gottfrid wrote:
> > Are there configuration suggestions for 1k nodes ? I was seeing tons
> > of timeouts trying to run 1k nodes. Are there network settings that I
> > need to make? Out of the box stuff seemed to work up to a couple
> > hundred but I want to go bigger.  Pointers/Suggestions?
> >
> > derek
> >
> >
> > ps: i wrote up my ec2/hadoop at nytimes.com - check it out
> > http://open.blogs.nytimes.com/2007/11/01/self-service-prorated-super-computing-fun/
>
>

Mime
View raw message