hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Runping Qi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3245) Provide ability to persist running jobs (extend HADOOP-1876)
Date Mon, 14 Apr 2008 20:27:06 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12588776#action_12588776

Runping Qi commented on HADOOP-3245:

bq There is one problem with this approach of periodic merges. We lose the information for
completed tasks that completes in the interval between the last merge and the time when the
JobTracker crashes. 

Are you suggest that you intend to use persisted job status as the basis for the job tracker
to started from where it crashed?
I don't think that is the jira 1876 was intended and its current implementation can support

I think we need to work out what is the intended behavior to re-start a job tracker, and what
needs to be persisted to support that behavior.
It wil be nice if the work for 1876 can be re-used. 
But at this time, I think it does not sound right to tweak 1876 for some functionalities we
don't have a clear consensus.

> Provide ability to persist running jobs (extend HADOOP-1876)
> ------------------------------------------------------------
>                 Key: HADOOP-3245
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3245
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: mapred
>            Reporter: Devaraj Das
>             Fix For: 0.18.0
> This could probably extend the work done in HADOOP-1876. This feature can be applied
for things like jobs being able to survive jobtracker restarts.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message