hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siqi Li (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (MAPREDUCE-6047) Introduce localization phase for mapreduce jobs
Date Fri, 22 Aug 2014 18:12:12 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-6047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Siqi Li reassigned MAPREDUCE-6047:
----------------------------------

    Assignee: Siqi Li

> Introduce localization phase for mapreduce jobs
> -----------------------------------------------
>
>                 Key: MAPREDUCE-6047
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6047
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Siqi Li
>            Assignee: Siqi Li
>
> we have been experienced lots of task timeout due to localization of a large number of
files. It would be great if we have an additional phase that keep track of how many files
have been localized. Thus for Mappers we had "localization", "map", "sort". For backwards
compatibility, we can keep it 0% but still heartbeat to AM with the status string "5/300 files
localized".



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message