Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Date: Tue, 9 Oct 2012 04:36:04 +0000 (UTC)
From: "Hudson (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: <1335744054.12815.1349757364149.JavaMail.jiratomcat@arcas>
In-Reply-To: <234841534.2070.1349386427604.JavaMail.jiratomcat@arcas>
Subject: [jira] [Commented] (MAPREDUCE-4705) Historyserver links expire
 before the history data does
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MAPREDUCE-4705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472122#comment-13472122 ] 

Hudson commented on MAPREDUCE-4705:
-----------------------------------

Integrated in Hadoop-Mapreduce-trunk-Commit #2856 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2856/])
    MAPREDUCE-4705. Fix a bug in job history lookup, which makes older jobs inaccessible despite the presence of a valid history file. (Contributed by Jason Lowe) (Revision 1395850)

     Result = FAILURE
sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1395850
Files : 
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobHistoryParsing.java

                
> Historyserver links expire before the history data does
> -------------------------------------------------------
>
>                 Key: MAPREDUCE-4705
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4705
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobhistoryserver, mrv2
>    Affects Versions: 0.23.3
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>            Priority: Critical
>             Fix For: 2.0.3-alpha, 0.23.5
>
>         Attachments: MAPREDUCE-4705.patch
>
>
> The historyserver can serve up links to jobs that become useless well before the job history files are purged.  For example on a large, heavily used cluster we can end up rotating through the maximum number of jobs the historyserver can track fairly quickly.  If a user was investigating an issue with a job using a saved historyserver URL, that URL can become useless because the historyserver has forgotten about the job even though the history files are still sitting in HDFS.
> We can tell the historyserver to keep track of more jobs by increasing {{mapreduce.jobhistory.joblist.cache.size}}, but this has a direct impact on the responsiveness of the main historyserver page since it serves up all the entries to the client at once.  It looks like Hadoop 1.x avoided this issue by encoding the history file location into the URLs served up by the historyserver, so it didn't have to track a mapping between job ID and history file location.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira