hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bhallamudi Venkata Siva Kamesh (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3971) Job History web services need to have limits on the number of itmes they can return.
Date Thu, 08 Mar 2012 13:47:57 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225205#comment-13225205
] 

Bhallamudi Venkata Siva Kamesh commented on MAPREDUCE-3971:
-----------------------------------------------------------

Hi Robert,
 As per my understanding, */mapreduce/jobs* API causes a very large load on the job history
server. However, a user can mention limit also, ofcourse it can be optional. 
 
 Intention of this ticket is to return only a limited number of items, something like recent
50 or 100 among 20000, in the absense of limit factor or irrespective of limit factor?

I think, suppose say a user mentions limit as 50, even then all the jobs from *intermediateListCache*
& *jobListCache* are being fetched from JHS and again iterating over the list in the HsWebServices,
to return only required number of jobs.  If so, I think, we can move the limit factor to JHS
to parse only that many jobs, some thing like following API will be useful

{noformat}
 public Map<JobId, Job> getNJobs(String queue, String user, long sTime, long sEnd, long
fStart, long fEnd, int n) {}
{noformat}

Please provide your comments.
                
> Job History web services need to have limits on the number of itmes they can return.
> ------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-3971
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3971
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: mrv2
>    Affects Versions: 0.23.2
>            Reporter: Robert Joseph Evans
>
> The Job History web services canput a very large load on the job history server.  We
should put in a limit on the number of entries that can be returned by the web service, and
also add in the ability to modify the starting location in the list, so that all entries can
still be downlaoded.  Just not all at once.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message