hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@apache.org>
Subject Re: Network problems Hadoop 0.20.2 and Terasort on Debian 2.6.32 kernel
Date Thu, 15 Apr 2010 16:09:50 GMT
Todd Lipcon wrote:
> On Tue, Apr 13, 2010 at 4:13 AM, stephen mulcahy
> <stephen.mulcahy@deri.org>wrote:

>> Sure, but I figured I'd go with a distro now that can be largely left
>> untouched for the next 2-3 years and Debian lenny felt that bit old for
>> that. I know RHEL/CentOS would fit that requirement also, will see. I'm also
>> interested in using DRBD in some of our nodes for redundancy, again, running
>> with a newer distro should reduce the pain of configuring that.
>>
>> Finally, I figured burning in our cluster was a good opportunity to give
>> back to the community and do some testing on their behalf.
>>
> 
> Very admirable of you :) It is good to have some people running new kernels
> to suss these issues out before the rest of us check out modern technology
> ;-)

Tom White is planning to split off a Hadoop 0.21 branch from SVN_TRUNK 
at the end of the month, so if you still want to do some cluster 
testing, he'd be grateful for that being tested on debian too

> 
> 
>> With regard to our TeraSort benchmark time of ~23 minutes - is that in the
>> right ballpark for a cluster of 45 data nodes and a nn and 2nn?

#of HDDs/server will be a factor too, and no, I don't know how to 
predict it.



Mime
View raw message