hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stephen mulcahy <stephen.mulc...@deri.org>
Subject Re: Slowdown with Hadoop Sort benchmark when using Jumbo frames?
Date Tue, 26 Jan 2010 18:12:23 GMT
Eli Collins wrote:
>> I was somewhat surprised at the results of enabling Jumbo frames - it
>> resulted in a slowdown. In the case of the write operations, the slow down
>> was about 5%. In the case of 10GB sort, the slowdown was around 6% and in
>> the case of the 100GB sort, the slowdown was nearly 20%.
> 
> Are normal (non-hadoop) file transfers faster between the same hosts
> with jumbo frames enabled? ie have your ruled out host/configuration
> issues?

Yes - I did some testing on two different clusters we have - both with 
and without jumbo frames and in both cases, I saw a small but noticeable 
increase in the overall bandwidth when moving from an MTU of 1500 to an 
MTU of 9000.

Has anyone else tested Hadoop performance with Jumbo frames? Are you 
seeing something different to what we're seeing?

Thanks,

-stephen

-- 
Stephen Mulcahy, DI2, Digital Enterprise Research Institute,
NUI Galway, IDA Business Park, Lower Dangan, Galway, Ireland
http://di2.deri.ie    http://webstar.deri.ie    http://sindice.com

Mime
View raw message