hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sriramadasu (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-4261) Jobs failing in the init stage will never cleanup
Date Wed, 15 Oct 2008 09:47:44 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-4261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Amareshwari Sriramadasu updated HADOOP-4261:
--------------------------------------------

    Release Note: A setup task (for doing OutputCommitter.setupJob()) will be launched, when
the job is in PREP state and after initializing tasks. Once the setup task completes, the
job will be moved to RUNNING state.
    Hadoop Flags: [Incompatible change, Reviewed]  (was: [Reviewed])

> Jobs failing in the init stage will never cleanup
> -------------------------------------------------
>
>                 Key: HADOOP-4261
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4261
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Amar Kamat
>            Assignee: Amareshwari Sriramadasu
>            Priority: Blocker
>             Fix For: 0.19.0
>
>         Attachments: patch-4261.txt, patch-4261.txt, patch-4261.txt, patch-4261.txt
>
>
> Pre HADOOP-3150, if the job fails in the init stage, {{job.kill()}} was called. This
used to make sure that the job was cleaned up w.r.t 
> - staus set to KILLED/FAILED
> - job files from the system dir are deleted
> - closing of job history files
> - making jobtracker aware of this through {{jobTracker.finalizeJob()}}
> - cleaning up the data structures via {{JobInProgress.garbageCollect()}}
> Now if the job fails in the init stage, {{job.fail()}} is called which doesnt do the
cleanup. HADOOP-3150 introduces cleanup tasks which are launched once the job completes i.e
killed/failed/succeeded.  Jobtracker will never consider this job for scheduling as the job
will be in the {{PREP}} state forever.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message