hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-5448) JobHistory should use JobTracker's configuration for paths created/read on hadoop.job.history.location
Date Wed, 11 Mar 2009 09:08:50 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Amar Kamat updated HADOOP-5448:
-------------------------------

    Attachment: HADOOP-5448-v1.2.patch

Attaching a patch that should fix the issue. This patch determines the FileSystem for {{hadoop.job.history.location}}
during init and uses this filesystem everywhere instead of computing it everytime. Result
of test-patch
{code}
 [exec] -1 overall.  
     [exec] 
     [exec]     +1 @author.  The patch does not contain any @author tags.
     [exec] 
     [exec]     -1 tests included.  The patch doesn't appear to include any new or modified
tests.
     [exec]                         Please justify why no tests are needed for this patch.
     [exec] 
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec] 
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler
warnings.
     [exec] 
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
     [exec] 
     [exec]     +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
     [exec] 
     [exec]     +1 release audit.  The applied patch does not increase the total number of
release audit warnings.
{code}

Test case is bit tricky as JobClient itself uses the job's fs info to upload job files. Ideally
the testcase should hack the jobclient code to upload files with faulty *fs.default.name*.

Note that some part of HADOOP-5328 is used here. 

> JobHistory should use JobTracker's configuration for paths created/read on hadoop.job.history.location
> ------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-5448
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5448
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Devaraj Das
>         Attachments: HADOOP-5448-v1.2.patch
>
>
> The JobHistory methods use the user's jobconf to create/read history file paths on both
hadoop.job.history.location and hadoop.job.history.user.location. Even the RecoveryManager
uses the user's jobconf. This should be fixed to use the JobTracker's conf for the history
files on hadoop.job.history.location.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message