hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1778) CompletedJobStatusStore initialization should fail if {mapred.job.tracker.persist.jobstatus.dir} is unwritable
Date Tue, 08 Jun 2010 13:57:11 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12876680#action_12876680

Amar Kamat commented on MAPREDUCE-1778:

bq. If the directory exists and permissions are correct jt can read from or write to - is
this not correct? 
How do you define the phrase "permissions are correct"?
- If the directory matches predefined and expected permissions
- if its writable and readable 

The good part about #2 is that no matter what the directory permissions, the verification
code is simply concerned with the job at hand i.e to test if the dir is r+w. #1 wont work
is completedjob-status-store dir is pre created with different permissions.

bq. As for test case, scope of JobStatusPersistency is different would rather have a separate
Krishna, there is a test scenario in TestJobStatusPersistency.java i.e testJobStoreDisablingWithInvalidPath,
which checks exactly the same thing as we intend to check here, except for the fact that instead
of silently disabling CompletedJobStatusStore, we should check if the jobtracker bails out.
TestJobStatusPersistency might sound as a misnomer but its exactly the feature provided by
the CompletedJobStatusStore.java.

> CompletedJobStatusStore initialization should fail if {mapred.job.tracker.persist.jobstatus.dir}
is unwritable
> --------------------------------------------------------------------------------------------------------------
>                 Key: MAPREDUCE-1778
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1778
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: jobtracker
>            Reporter: Amar Kamat
>            Assignee: Amar Kamat
>         Attachments: mapred-1778-1.patch, mapred-1778-2.patch, mapred-1778.patch
> If {mapred.job.tracker.persist.jobstatus.dir} points to an unwritable location or mkdir
of {mapred.job.tracker.persist.jobstatus.dir} fails, then CompletedJobStatusStore silently
ignores the failure and disables CompletedJobStatusStore. Ideally the JobTracker should bail
out early indicating a misconfiguration.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message