hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-6962) ACLs inheritance conflict with umaskmode
Date Fri, 14 Nov 2014 18:48:35 GMT

     [ https://issues.apache.org/jira/browse/HDFS-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chris Nauroth updated HDFS-6962:
--------------------------------
    Target Version/s: 2.7.0  (was: 2.4.1)

Hello, [~Alexandre LINTE].  Thank you for filing this issue.  I tested the same scenario against
a Linux local file system, and I confirmed that HDFS is showing different behavior, just like
you described.

I also confirmed that this is a divergence from the POSIX ACL specs.  Here is a quote of the
relevant section:

{quote}
The permissions of inherited access ACLs are further modified by the mode parameter that each
system call creating file system objects has. The mode parameter contains nine permission
bits that stand for the permissions of the owner, group, and other class permissions. The
effective permissions of each class are set to the intersection of the permissions defined
for this class in the ACL and specified in the mode parameter.

If the parent directory has no default ACL, the permissions of the new file are determined
as defined in POSIX.1. The effective permissions are set to the permissions defined in the
mode parameter, minus the permissions set in the current umask.

The umask has no effect if a default ACL exists.
{quote}

Changing this behavior is going to be somewhat challenging.  Note the distinction made in
the spec between mode and umask.  When creating a new child (file or directory) of a directory
with a default ACL, the mode influences the inherited access ACL entries, but the umask has
no effect.  Unfortunately, our current implementation intersects mode and umask on the client
side before passing them to the NameNode in the RPC.  This happens in {{DFSClient#mkdirs}}
and {{DFSClient#create}}:

{code}
  public boolean mkdirs(String src, FsPermission permission,
      boolean createParent) throws IOException {
    if (permission == null) {
      permission = FsPermission.getDefault();
    }
    FsPermission masked = permission.applyUMask(dfsClientConf.uMask);
{code}

{code}
  public DFSOutputStream create(String src, 
                             FsPermission permission,
                             EnumSet<CreateFlag> flag, 
                             boolean createParent,
                             short replication,
                             long blockSize,
                             Progressable progress,
                             int buffersize,
                             ChecksumOpt checksumOpt,
                             InetSocketAddress[] favoredNodes) throws IOException {
    checkOpen();
    if (permission == null) {
      permission = FsPermission.getFileDefault();
    }
    FsPermission masked = permission.applyUMask(dfsClientConf.uMask);
{code}

On the NameNode side, when it copies the default ACL from parent to child, we've lost the
information.  We just have a single piece of permissions data, with no knowledge of what was
the mode vs. the umask on the client side.

A potential solution is to push both mode and umask explicitly to the NameNode in the RPC
requests for {{MkdirsRequestProto}} and {{CreateRequestProto}}.  Those messages already contain
an instance of {{FsPermissionProto}}.  We could add a second optional instance.  If both instances
are defined, then the NameNode would interpret one as being mode and the other as being umask.
 There would still be a possibility of an older client still passing just one instance, and
in that case, we'd have to fall back to the current behavior.  It's a bit messy, but it could
work.

We also have one additional problem specific to the shell for files (not directories).  The
implementation of copyFromLocal breaks down into 2 separate RPCs: creating the file, followed
by a separate chmod call.  The NameNode has no way of knowing if that chmod call is part of
a copyFromLocal or not though.  It's too late to enforce the mode vs. umask distinction.

I'm tentatively targeting this to 2.7.0.  I think this will need more investigation to make
sure there are no compatibility issues with the solution.  If there is an unavoidable compatibility
problem, then it might require pushing out to 3.x.  We won't know for sure until someone starts
coding.

Thank you again for the very detailed bug report.

> ACLs inheritance conflict with umaskmode
> ----------------------------------------
>
>                 Key: HDFS-6962
>                 URL: https://issues.apache.org/jira/browse/HDFS-6962
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: security
>    Affects Versions: 2.4.1
>         Environment: CentOS release 6.5 (Final)
>            Reporter: LINTE
>              Labels: hadoop, security
>
> In hdfs-site.xml 
> <property>
>     <name>dfs.umaskmode</name>
>     <value>027</value>
> </property>
> 1/ Create a directory as superuser
> bash# hdfs dfs -mkdir  /tmp/ACLS
> 2/ set default ACLs on this directory rwx access for group readwrite and user toto
> bash# hdfs dfs -setfacl -m default:group:readwrite:rwx /tmp/ACLS
> bash# hdfs dfs -setfacl -m default:user:toto:rwx /tmp/ACLS
> 3/ check ACLs /tmp/ACLS/
> bash# hdfs dfs -getfacl /tmp/ACLS/
> # file: /tmp/ACLS
> # owner: hdfs
> # group: hadoop
> user::rwx
> group::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> user::rwx | group::r-x | other::--- matches with the umaskmode defined in hdfs-site.xml,
everything ok !
> default:group:readwrite:rwx allow readwrite group with rwx access for inhéritance.
> default:user:toto:rwx allow toto user with rwx access for inhéritance.
> default:mask::rwx inhéritance mask is rwx, so no mask
> 4/ Create a subdir to test inheritance of ACL
> bash# hdfs dfs -mkdir  /tmp/ACLS/hdfs
> 5/ check ACLs /tmp/ACLS/hdfs
> bash# hdfs dfs -getfacl /tmp/ACLS/hdfs
> # file: /tmp/ACLS/hdfs
> # owner: hdfs
> # group: hadoop
> user::rwx
> user:toto:rwx   #effective:r-x
> group::r-x
> group:readwrite:rwx     #effective:r-x
> mask::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> Here we can see that the readwrite group has rwx ACL bu only r-x is effective because
the mask is r-x (mask::r-x) in spite of default mask for inheritance is set to default:mask::rwx
on /tmp/ACLS/
> 6/ Modifiy hdfs-site.xml et restart namenode
> <property>
>     <name>dfs.umaskmode</name>
>     <value>010</value>
> </property>
> 7/ Create a subdir to test inheritance of ACL with new parameter umaskmode
> bash# hdfs dfs -mkdir  /tmp/ACLS/hdfs2
> 8/ Check ACL on /tmp/ACLS/hdfs2
> bash# hdfs dfs -getfacl /tmp/ACLS/hdfs2
> # file: /tmp/ACLS/hdfs2
> # owner: hdfs
> # group: hadoop
> user::rwx
> user:toto:rwx   #effective:rw-
> group::r-x      #effective:r--
> group:readwrite:rwx     #effective:rw-
> mask::rw-
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> So HDFS masks the ACL value (user, group and other  -- exepted the POSIX owner -- ) with
the group mask of dfs.umaskmode properties when creating directory with inherited ACL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message