hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4372) Improve the way the job history files are managed during job recovery
Date Thu, 21 May 2009 09:03:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711523#action_12711523
] 

Devaraj Das commented on HADOOP-4372:
-------------------------------------

I committed this to 0.20 branch as well. There have been problems like job submission taking
a long time when the number of files in the history folder are too large. This patch introduced
an API that would fasten the process of obtaining a history file for a new job (earlier it
used to do a scan of the history folder which is costly when the number of files are large).

> Improve the way the job history files are managed during job recovery
> ---------------------------------------------------------------------
>
>                 Key: HADOOP-4372
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4372
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Amar Kamat
>            Assignee: Amar Kamat
>             Fix For: 0.21.0
>
>         Attachments: HADOOP-4372-v1.4.patch, HADOOP-4372-v1.patch, HADOOP-4372-v3.0.patch,
HADOOP-4372-v3.1.patch
>
>
> Today we use the _.recover_ technique to handle the job history files when the jobtracker
restarts. The comment [here|https://issues.apache.org/jira/browse/HADOOP-3245?focusedCommentId=12629080#action_12629080]
proposes a better way to handle the files. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message