hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-195) transfer map output transfer with http instead of rpc
Date Tue, 09 May 2006 18:40:05 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-195?page=comments#action_12378685 ] 

Owen O'Malley commented on HADOOP-195:
--------------------------------------

As to why we lost the 20 reduces, here is the breakdown:

no progress update to task tracker for 20 min: 11
rpc timeout on progress: 4
lost task tracker (no heartbeat to job tracker for 10 min): 3
exit 65 (caused by ping failures): 4

one task had exit 65 and lost task tracker
one task had exit 65 and no progress update

so it task tracker latency is still a big concern.

> transfer map output transfer with http instead of rpc
> -----------------------------------------------------
>
>          Key: HADOOP-195
>          URL: http://issues.apache.org/jira/browse/HADOOP-195
>      Project: Hadoop
>         Type: Improvement

>   Components: mapred
>     Versions: 0.2
>     Reporter: Owen O'Malley
>     Assignee: Owen O'Malley
>      Fix For: 0.3
>  Attachments: netstat.log, netstat.xls
>
> The data transfer of the map output should be transfered via http instead rpc, because
rpc is very slow for this application and the timeout behavior is suboptimal. (server sends
data and client ignores it because it took more than 10 seconds to be received.)

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message