hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anupam Seth (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-2187) map tasks timeout during sorting
Date Mon, 01 Aug 2011 22:17:48 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-2187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Anupam Seth updated MAPREDUCE-2187:
-----------------------------------

    Attachment: MAPREDUCE-2187-trunk-v3.patch

> map tasks timeout during sorting
> --------------------------------
>
>                 Key: MAPREDUCE-2187
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2187
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 0.20.2, 0.20.205.0
>            Reporter: Gianmarco De Francisci Morales
>            Assignee: Anupam Seth
>             Fix For: 0.20.205.0
>
>         Attachments: MAPREDUCE-2187-20-security-v2.patch, MAPREDUCE-2187-20-security.patch,
MAPREDUCE-2187-22.patch, MAPREDUCE-2187-MR-279-v2.patch, MAPREDUCE-2187-branch-MR-279.patch,
MAPREDUCE-2187-trunk-v2.patch, MAPREDUCE-2187-trunk-v3.patch, MAPREDUCE-2187-trunk.patch
>
>
> During the execution of a large job, the map tasks timeout:
> {code}
> INFO mapred.JobClient: Task Id : attempt_201010290414_60974_m_000057_1, Status : FAILED
> Task attempt_201010290414_60974_m_000057_1 failed to report status for 609 seconds. Killing!
> {code}
> The bug is in the fact that the mapper has already finished, and, according to the logs,
the timeout occurs during the merge sort phase.
> The intermediate data generated by the map task is quite large. So I think this is the
problem.
> The logs show that the merge-sort was running for 10 minutes when the task was killed.
> I think the mapred.Merger should call Reporter.progress() somewhere.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message