hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4039) Retired jobs are not present in the job list returned to the job-client.
Date Tue, 02 Sep 2008 16:23:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12627729#action_12627729

Owen O'Malley commented on HADOOP-4039:

    When the number of jobs could get very long, the second option seems better - less data
from JT to client, accomodates all the above types of listing jobs in a single RPC, for e.g.,
we then won't need a separate jobsToComplete. Thoughts?

We need to design all of the protocols so that it is easy to page through them without duplicates.
In the task events we did it via:

TaskCompletionEvent[] getTaskCompletionEvents(JobID jobid, int fromEventId,
       int maxEvents) throws IOException;

for job status, we need to use time:

JobStatus[] getJobStatus(long minSubmitDate, int maxJobs) throws IOException;

Note that we'd also need to add the submit date to the JobStatus so that you can page through
them with the client. Thoughts?

> Retired jobs are not present in the job list returned to the job-client.
> ------------------------------------------------------------------------
>                 Key: HADOOP-4039
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4039
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Vinod Kumar Vavilapalli
> After {{mapred.jobtracker.retirejob.interval}} elapses, completed jobs are no longer
maintained by the JT, but instead when job-client ask for job status, counters or task completion
events, the relevant information is picked up from completed job store. But a retired job
is not listed in the output of "{{hadoop job -list all}}", without which other information
from completed job store isn't quite useful unless a job-id is known from elsewhere.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message