hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From yang song <hadoop.ini...@gmail.com>
Subject How to speed up the copy phrase?
Date Mon, 24 Aug 2009 06:49:42 GMT
Hello, everyone

When I submit a big job(e.g. maptasks:10000, reducetasks:500), I find that
the copy phrase will last for a long long time. From WebUI, the message
"reduce > copy (xxxx of 10000 at 0.01 MB/s) >" tells me the transfer speed
is just 0.01 MB/s. Does it a regular value? How can I solve it?

Thank you!

P.S. The hadoop version is 0.19.1. The cluster has 20 nodes. Heap size of JT
is 6G while the     others are default settings.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message