hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jakob Homan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-760) "fs -put" fails if dfs.umask is set to 63
Date Wed, 18 Nov 2009 03:04:39 GMT

    [ https://issues.apache.org/jira/browse/HDFS-760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12779270#action_12779270

Jakob Homan commented on HDFS-760:

Some info: This is being caused by a problem in the configuration.  The stack trace: 
java.lang.IllegalArgumentException: 63
        at org.apache.hadoop.fs.permission.PermissionParser.<init>(PermissionParser.java:54)
        at org.apache.hadoop.fs.permission.UmaskParser.<init>(UmaskParser.java:37)
        at org.apache.hadoop.fs.permission.FsPermission.getUMask(FsPermission.java:204)
        at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:569)
        at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:537)
        at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:213)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:547)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:528)
        at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:435)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:219)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:192)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:156)
        at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1533)
        at org.apache.hadoop.fs.FsShell.copyFromLocal(FsShell.java:129)
        at org.apache.hadoop.fs.FsShell.run(FsShell.java:1837)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at org.apache.hadoop.fs.FsShell.main(FsShell.java:1974)
is misleading.
dfs.umask is the old-style key for setting the umask and is correctly read in by the Configuration
as the old-style. However, the section of the code that determines whether or not to treat
it as an old-style, decimal value (that should be converted to octal before being passed to
the PermissionParser) is being given the wrong answer by the configuration:
    if(conf != null) {
      String confUmask = conf.get(UMASK_LABEL);
      if(confUmask != null)  // UMASK_LABEL is set
        if(conf.deprecatedKeyWasSet(DEPRECATED_UMASK_LABEL)) { // <--- this is returning
false but should be true
          umask = Integer.parseInt(confUmask); // Evaluate as decimal value
          umask = new UmaskParser(confUmask).getUMask();  // <----- therefore this tries
to parse the decimal as padded octal and fails
The code in Configuration that checks whether or not a deprecated value was set is returning
false, though it shouldn't be (specifically deprecatedKeyMap.get(oldKey).accessed is still
set to false and should be true).  I'll look more tomorrow.

Regardless of the reason, we should probably have better exception handling in the FsShell.
 The exception thrown from PermissionParser should be more descriptive so that when it hits
the user there is a better sense of what went wrong.

> "fs -put" fails if dfs.umask is set to 63
> -----------------------------------------
>                 Key: HDFS-760
>                 URL: https://issues.apache.org/jira/browse/HDFS-760
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 0.21.0
>            Reporter: Tsz Wo (Nicholas), SZE
>            Priority: Blocker
>             Fix For: 0.21.0, 0.22.0
> Add the following to hdfs-site.conf
> {noformat}
>   <property>
>     <name>dfs.umask</name>
>     <value>63</value>
>   </property>
> {noformat}
> Then run "hadoop fs -put"
> {noformat}
> -bash-3.1$ ./bin/hadoop fs -put README.txt r.txt
> 09/11/09 23:09:07 WARN conf.Configuration: mapred.task.id is deprecated. Instead, use
> put: 63
> Usage: java FsShell [-put <localsrc> ... <dst>]
> -bash-3.1$
> {noformat}
> Observed the above behavior in 0.21.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message