hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4705) Historyserver links expire before the history data does
Date Mon, 08 Oct 2012 21:26:02 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13471871#comment-13471871

Hadoop QA commented on MAPREDUCE-4705:

{color:green}+1 overall{color}.  Here are the results of testing the latest attachment 
  against trunk revision .

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 1 new or modified
test files.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 javadoc{color}.  The javadoc tool did not generate any warning messages.

    {color:green}+1 eclipse:eclipse{color}.  The patch built with eclipse:eclipse.

    {color:green}+1 findbugs{color}.  The patch does not introduce any new Findbugs (version
1.3.9) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:green}+1 core tests{color}.  The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs.

    {color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2916//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2916//console

This message is automatically generated.
> Historyserver links expire before the history data does
> -------------------------------------------------------
>                 Key: MAPREDUCE-4705
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4705
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobhistoryserver, mrv2
>    Affects Versions: 0.23.3
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>            Priority: Critical
>         Attachments: MAPREDUCE-4705.patch
> The historyserver can serve up links to jobs that become useless well before the job
history files are purged.  For example on a large, heavily used cluster we can end up rotating
through the maximum number of jobs the historyserver can track fairly quickly.  If a user
was investigating an issue with a job using a saved historyserver URL, that URL can become
useless because the historyserver has forgotten about the job even though the history files
are still sitting in HDFS.
> We can tell the historyserver to keep track of more jobs by increasing {{mapreduce.jobhistory.joblist.cache.size}},
but this has a direct impact on the responsiveness of the main historyserver page since it
serves up all the entries to the client at once.  It looks like Hadoop 1.x avoided this issue
by encoding the history file location into the URLs served up by the historyserver, so it
didn't have to track a mapping between job ID and history file location.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message