hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From elton sky <eltonsky9...@gmail.com>
Subject Why inter-rack communication in mapreduce slow?
Date Mon, 06 Jun 2011 07:22:58 GMT
hello everyone,

As I don't have experience with big scale cluster, I cannot figure out why
the inter-rack communication in a mapreduce job is "significantly" slower
than intra-rack.
I saw cisco catalyst 4900 series switch can reach upto 320Gbps forwarding
capacity. Connected with 48 nodes with 1Gbps ethernet each, it should not be
much contention at the switch, is it?

Can anyone explain how this "slow"ness happens to me?

Thanks
Elton

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message