hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1778) CompletedJobStatusStore initialization should fail if {mapred.job.tracker.persist.jobstatus.dir} is unwritable
Date Mon, 31 May 2010 02:49:36 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12873530#action_12873530

Amar Kamat commented on MAPREDUCE-1778:

There are 2 cases here
1) If mkdir() fails
2) If mapred.job.tracker.persist.jobstatus.dir exists but is not writable

We need to fix both these cases. From the [latest patch|http://tinyurl.com/294qyjr], I see
that you have fixed #1. For #2, I think maybe we can try and create a dummy temp file. Is
there a better way than this? Did not find any helper api in FileSystem.

Comments on the testcases :
1) Can you kindly remove commented code and add javadoc comments for the testcases. 
2) Instead of adding a new testcase, its better that we modify the existing testcase  i.e
TestJobStatusPersistency. Here you can change testJobStoreDisablingWithInvalidPath() (rename
it to testJobStoreWithInvalidPath)  to fail when an invalid path is provided instead of disabling.
You can add a new test (say testJobStoreWithUnwritablePath()) for testing #2.

> CompletedJobStatusStore initialization should fail if {mapred.job.tracker.persist.jobstatus.dir}
is unwritable
> --------------------------------------------------------------------------------------------------------------
>                 Key: MAPREDUCE-1778
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1778
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: jobtracker
>            Reporter: Amar Kamat
>            Assignee: Amar Kamat
>         Attachments: mapred-1778-1.patch, mapred-1778.patch
> If {mapred.job.tracker.persist.jobstatus.dir} points to an unwritable location or mkdir
of {mapred.job.tracker.persist.jobstatus.dir} fails, then CompletedJobStatusStore silently
ignores the failure and disables CompletedJobStatusStore. Ideally the JobTracker should bail
out early indicating a misconfiguration.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message