hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Runping Qi (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-322) Need a job control utility to submit and monitor a group of jobs which have DAG dependency
Date Sat, 24 Jun 2006 05:15:29 GMT
Need a job control utility to submit and monitor a group of jobs which have DAG dependency
------------------------------------------------------------------------------------------

         Key: HADOOP-322
         URL: http://issues.apache.org/jira/browse/HADOOP-322
     Project: Hadoop
        Type: New Feature

    Reporter: Runping Qi
 Assigned to: Runping Qi 



In my applications, some jobs depend on the outputs of other jobs. Therefore, job dependency
forms a DAG. A job is ready to run if and only if it does not have any dependency or all the
jobs it depends are finished successfully. To help schedule and monitor a group of jobs like
that, I am thinking of implementing a utility that:
	- accept jobs with dependency specification
      - monitor job status
      - submit jobs when they are ready

With such a utility, the application can construct its jobs, specify their dependency and
then hand the jobs to the utility class. The utility takes care of the details of job submission.

I'll post my design skech for comments/suggestion.
Eventually, I'll submit a patch for the utility.





-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message