hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1778) CompletedJobStatusStore initialization should fail if {mapred.job.tracker.persist.jobstatus.dir} is unwritable
Date Fri, 11 Jun 2010 03:01:16 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12877697#action_12877697
] 

Amar Kamat commented on MAPREDUCE-1778:
---------------------------------------

bq. I only see FileStatus.isDir() 
Look at [FileStatus.java|http://tinyurl.com/2uaew2a] from hadoop-common.

Comments:
# I still see few lines of code crossing the 80 column margin. 
# Plz add comments. For example 
{code}
+      } else {
+        FileStatus stat = fs.getFileStatus(path);
+        FsPermission actual = stat.getPermission();
+        if (!stat.isDir())
+          throw new DiskErrorException("not a directory: " 
+                                   + path.toString());
+        FsAction user = actual.getUserAction();
+        if (!user.implies(FsAction.READ))
+          throw new DiskErrorException("directory is not readable: "
+                                   + path.toString());
+        if (!user.implies(FsAction.WRITE))
+          throw new DiskErrorException("directory is not writable: "
+                                   + path.toString());
{code} 
It would be nice to explain why its done this way. 

Testcase:
# It would be better if we test the following scenarios 
 ## directory doesnt exist but mkdir fails() [example: parent folder doesnt have write permissions?]
 ## directory exists but no read/write permissions
 ## completedjob status store is a file and not a directory
I think the current changes in the patch tests sceanrio#2. Can you please add tests for testing
scenario#1 and scenario#3?
# If the JobTracker gets started, it should be shut down. 
# For a newly created job-status-store dir, it would be nice to validate its permissions (i.e
JOB_STATUS_STORE_DIR_PERMISSION).

> CompletedJobStatusStore initialization should fail if {mapred.job.tracker.persist.jobstatus.dir}
is unwritable
> --------------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1778
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1778
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: jobtracker
>            Reporter: Amar Kamat
>            Assignee: Krishna Ramachandran
>         Attachments: mapred-1778-1.patch, mapred-1778-2.patch, mapred-1778-3.patch, mapred-1778-4.patch,
mapred-1778.patch
>
>
> If {mapred.job.tracker.persist.jobstatus.dir} points to an unwritable location or mkdir
of {mapred.job.tracker.persist.jobstatus.dir} fails, then CompletedJobStatusStore silently
ignores the failure and disables CompletedJobStatusStore. Ideally the JobTracker should bail
out early indicating a misconfiguration.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message