hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sriramadasu (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-3327) Shuffling fetchers waited too long between map output fetch re-tries
Date Fri, 16 Jan 2009 11:49:59 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-3327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Amareshwari Sriramadasu updated HADOOP-3327:
--------------------------------------------

    Attachment: patch-3327.txt

Attaching a patch for review, while i continue my testing.

I have done simple tests with patch. Results are as follows:
With read timeouts simulated for 4 map outputs :
 *  On Single node cluster with 2 maps, 1 reducer  and maxMapRuntime 6sec.
    ** Job took 41 mins 4sec without the patch
    ** Job took 22mins 2 sec with the patch
* 20 node cluster with 50 maps, 6 reducers and maxMapRuntime 6sec
    ** Job took 18 mins 23sec without the patch
    ** Job took 6mins 32sec with the patch.
* 20 node cluster with 50 maps, 6 reducers with maxMapRuntime 2 mins 33 sec
    ** Job took 30mins 0sec without the patch
    ** Job took 8mins 30sec with the patch


> Shuffling fetchers waited too long between map output fetch re-tries
> --------------------------------------------------------------------
>
>                 Key: HADOOP-3327
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3327
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Runping Qi
>            Assignee: Amareshwari Sriramadasu
>         Attachments: hadoop-3327-v1.patch, hadoop-3327-v2.patch, hadoop-3327-v3.patch,
hadoop-3327.patch, patch-3327.txt
>
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message