hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3245) Provide ability to persist running jobs (extend HADOOP-1876)
Date Fri, 22 Aug 2008 10:28:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12624644#action_12624644
] 

Amar Kamat commented on HADOOP-3245:
------------------------------------

One more issue that needs to be addressed is trashing . When the jobtracker restarts, it will
recover logged tasks and schedule the rest. Some trackers that join early might get a task
which is running on a tracker that has not yet joined. Under such a case both the attempts
will run in parallel and the task that finishes first will kill the other. The problem with
this is that the slots will be wasted. Also this will add to the job runtime if the tasks
are long running. Some delay in opening the scheduling window might help. It looks like a
minor issue for now and can be handled in a separate issue.

> Provide ability to persist running jobs (extend HADOOP-1876)
> ------------------------------------------------------------
>
>                 Key: HADOOP-3245
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3245
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: mapred
>            Reporter: Devaraj Das
>            Assignee: Amar Kamat
>         Attachments: HADOOP-3245-v2.5.patch, HADOOP-3245-v2.6.5.patch, HADOOP-3245-v2.6.9.patch,
HADOOP-3245-v4.1.patch, HADOOP-3245-v5.13.patch, HADOOP-3245-v5.14.patch, HADOOP-3245-v5.26.patch
>
>
> This could probably extend the work done in HADOOP-1876. This feature can be applied
for things like jobs being able to survive jobtracker restarts.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message