Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <63509301.1227511784340.JavaMail.jira@brutus>
Date: Sun, 23 Nov 2008 23:29:44 -0800 (PST)
From: "Jothi Padmanabhan (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-4714) map tasks timing out during merge
 phase
In-Reply-To: <519168781.1227508784154.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-4714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12650126#action_12650126 ] 

Jothi Padmanabhan commented on HADOOP-4714:
-------------------------------------------

It could also be possible that we have lesser records in each spill file because each record is of a bigger size. Since the limits for spill files are by size (io.sort.mb) and not record count, we probably are not touching the 10,000 mark at all?

Christian, could you tell us what would be the approximate/average record size of the intermediate map output and the io.sort.mb in this case?


> map tasks timing out during merge phase
> ---------------------------------------
>
>                 Key: HADOOP-4714
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4714
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.18.1
>            Reporter: Christian Kunz
>
> With compression of transient data turned on some parts of the merge phase seem to not report progress enough.
> We see a lot of task failures during the merge phase, most of them timing out (even with a 20 min timeout)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.