hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sreekanth Ramakrishnan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5048) Sometimes job is still displayed in jobqueue_details page for long time after job was killed.
Date Fri, 16 Jan 2009 03:42:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12664397#action_12664397

Sreekanth Ramakrishnan commented on HADOOP-5048:

In case of zero tasks assigned to a job. The job's initTask() is called. Then in next polling
cycle the list of JobInProgress on which initTasks() has been called is checked and the finished
jobs are removed from wait queue.

With regards to comment on [HADOOP-4513|http://issues.apache.org/jira/browse/HADOOP-4513?focusedCommentId=12648951#action_12648951]
the reason why that was not implemented as follows:

In previous implementation, the maintaining the list of jobs which has been passed to Init-thread
workers were done lazily while walking thro' waiting job queue. There was no separate walk
done over the list of jobs which has been passed over init-thread. If JobQueuesManager had
taken responsibility of removing jobs from waiting queue, then our list will grown indefinitely.
This was the reason which it was not done.

Now, instead we have two walks:

* Clean up jobs in the list of jobs passed to init-threads.
* Walk thro' the wait queue until you meet maximum number of jobs to be initialized in a job
queue (an optimization so that we need not walk thro' entire wait queue)

> Sometimes job is still displayed in jobqueue_details page for long time after job was
> ---------------------------------------------------------------------------------------------
>                 Key: HADOOP-5048
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5048
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: contrib/capacity-sched
>            Reporter: Karam Singh
>            Assignee: Sreekanth Ramakrishnan
>         Attachments: HADOOP-5048-1.patch
> When I tried kill all running job, I noticed that were two jobs were listed on jobqueue_details.jsp
page page as well as they were also listed under failed job on jobtracker.jsp page.
> When I checked status of each that was displayed "killed" and Cleanup task status as
"Successful", but both jobs were also being on jobqueue_details.jsp page for longtime e.g
up to 10 -15 mins after I restarted JobTracker.
> Before killing the jobs, status of both jobs was running and no task of from them was
> I noticed this behavior on 3 different occasions. But is this random, not always reproducible.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message