Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Message-ID: <18030838.94201289863395276.JavaMail.jira@thor>
Date: Mon, 15 Nov 2010 18:23:15 -0500 (EST)
From: "Gianmarco De Francisci Morales (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Subject: [jira] Commented: (MAPREDUCE-2187) map tasks timeout during sorting
In-Reply-To: <27445467.86651289842213981.JavaMail.jira@thor>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MAPREDUCE-2187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12932251#action_12932251 ] 

Gianmarco De Francisci Morales commented on MAPREDUCE-2187:
-----------------------------------------------------------

No combiner and 200 reducers when I tried.
Raising the number of mappers and reducers makes the issue disappear.
But it is not a solution, I think.

I can give more details on the job if needed (number of records, size of the output, etc...)

> map tasks timeout during sorting
> --------------------------------
>
>                 Key: MAPREDUCE-2187
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2187
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 0.20.2
>            Reporter: Gianmarco De Francisci Morales
>
> During the execution of a large job, the map tasks timeout:
> {code}
> INFO mapred.JobClient: Task Id : attempt_201010290414_60974_m_000057_1, Status : FAILED
> Task attempt_201010290414_60974_m_000057_1 failed to report status for 609 seconds. Killing!
> {code}
> The bug is in the fact that the mapper has already finished, and, according to the logs, the timeout occurs during the merge sort phase.
> The intermediate data generated by the map task is quite large. So I think this is the problem.
> The logs show that the merge-sort was running for 10 minutes when the task was killed.
> I think the mapred.Merger should call Reporter.progress() somewhere.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.