hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sriramadasu (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-3150) Move task file promotion into the task
Date Tue, 02 Sep 2008 09:46:44 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Amareshwari Sriramadasu updated HADOOP-3150:
--------------------------------------------

    Attachment: patch-3150.txt

Here is a patch incorporating Devaraj's comments.

The patch has been tested for the following (with speculative execution on):
1. Applications creating side files
2. Lost task tracker with commit pending tasks in it.
3. Lost task tracker with commit approval received task on it.
4. Task failed after getting the commit approval.
5. Task failed after doing the commit.
6. Killing speculative commit pending task if another task has already got the commit approval.
7. Commit Response missed from the job tracker in the heartbeat response
8. Failure in cleanup task.
9. Four attempts of cleanup task failure leading to job failure.

Ran sort benchmark on big cluster.

> Move task file promotion into the task
> --------------------------------------
>
>                 Key: HADOOP-3150
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3150
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Owen O'Malley
>            Assignee: Amareshwari Sriramadasu
>             Fix For: 0.19.0
>
>         Attachments: 3150.patch, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt,
patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt,
patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt,
patch-3150.txt, patch-3150.txt, patch-3150.txt, patch-3150.txt
>
>
> We need to move the task file promotion from the JobTracker to the Task and move it down
into the output format.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message