hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ahmed Radwan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4346) Adding a refined version of JobTracker.getAllJobs() and exposing through the JobClient
Date Wed, 27 Jun 2012 08:03:44 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13402047#comment-13402047
] 

Ahmed Radwan commented on MAPREDUCE-4346:
-----------------------------------------

As I highlighted in the ticket description above: The JobClient only exposes a getAllJobs()
which returns all submitted jobs in any state, the result also includes all retired jobs.
This list is long and represents an unneeded overhead especially in the case of clients only
interested in jobs in specific states. 

One use case is a monitoring service that uses the JobClient and periodically calls getAllJobs()
to keep track of submitted jobs. Just using the current getAllJobs() will represent a communication
overhead because the returned list is unnecessarily long with redundant information (when
called periodically).

The new api provides a way for clients to selectively filter the long list which is normally
returned by getAllJobs(). The Client can now specify as part of the call: the job statuses
of interest and if including retired jobs is desired or not.

What do you think Arun?
                
> Adding a refined version of JobTracker.getAllJobs() and exposing through the JobClient
> --------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-4346
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4346
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: mrv1
>            Reporter: Ahmed Radwan
>            Assignee: Ahmed Radwan
>         Attachments: MAPREDUCE-4346.patch, MAPREDUCE-4346_rev2.patch, MAPREDUCE-4346_rev3.patch,
MAPREDUCE-4346_rev4.patch
>
>
> The current implementation for JobTracker.getAllJobs() returns all submitted jobs in
any state, in addition to retired jobs. This list can be long and represents an unneeded overhead
especially in the case of clients only interested in jobs in specific state(s). 
> It is beneficial to include a refined version where only jobs having specific statuses
are returned and retired jobs are optional to include. 
> I'll be uploading an initial patch momentarily.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message