hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11321) copyToLocal cannot save a file to an SMB share unless the user has Full Control permissions.
Date Tue, 02 Dec 2014 20:55:13 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14232111#comment-14232111

Colin Patrick McCabe commented on HADOOP-11321:

bq. Actually, it's more confusing than that, because we're not consistent about it. It looks
like file creations always apply fs.permissions.umask-mode. For directories, FileContext applies
it, but FileSystem doesn't. This means, for example, that "hadoop fs -mkdir" on the local
file system is in fact governed by the process umask.

This seems like a bug in {{FileSystem}}-- can you open a follow-on JIRA?  It may be that we
can only make the fix in 3.0, but we should still fix it.

bq. Another interesting thing I found after further experimentation is that the problem does
not repro for an SMB share mounted on Linux. A chmod call "succeeds" without error, but simply
does not change the permissions. The error handling seems to be specific to the OS client.
This may in fact turn out to be a Windows-only bug, contrary to my prior statement.

This still seems like incorrect behavior to me.  If someone asks us for a restrictive set
of permissions and we create the file with a more relaxed set, that seems insecure.

bq. I'm not aware of any Unix file/directory creation syscalls that let you bypass the umask.
That would mean achieving atomic create-and-set-permissions would require a native umask(0)
call. I'm very reluctant to do that, because we can't predict how this might compromise existing
applications, especially for applications that use a mix of Hadoop and their own file creation
calls. I suppose another possibility is to fork another process to do its own umask(0), but
then we'd have a lot of process creation overhead.

I think maybe the right thing to do is to issue a {{LOG.info}} if the process umask doesn't
match the configured value of {{fs.permissions.umask-mode}}.  Most users are probably running
with a umask that matches {{fs.permissions.umask-mode}} anyway.  If they aren't, the only
problem is that the LocalFilesystem will create files with more restrictive permissions than
intended.  This isn't a security hole, just a hassle, since things are just more locked down
than intended.  And it should be easy to fix if the user reads the logs and realizes his umask
is wrong.

bq. Considering all of that, I'm currently pursuing a Windows-only native code implementation,
with Linux continuing to run the existing code path. I believe this can work, because Windows
does not have a process umask or anything equivalent that would interfere with the intention
of fs.permissions.umask-mode. Unfortunately, creations on Linux would still be subject to
the race condition between creat/mkdir and chmod that we have in today's code, but at least
the situation wouldn't get any worse.

If you want to just do the Windows part here, I am +1 on that.  But I think we should also
do this for Linux.  We should eliminate this race condition on all platforms.  So if you choose
to just do Windows here, I'll open a follow-on JIRA for Linux.

Thanks, [~cnauroth]!

> copyToLocal cannot save a file to an SMB share unless the user has Full Control permissions.
> --------------------------------------------------------------------------------------------
>                 Key: HADOOP-11321
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11321
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 2.6.0
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
>         Attachments: HADOOP-11321.1.patch, HADOOP-11321.2.patch, winutils.tmp.patch
> In Hadoop 2, it is impossible to use {{copyToLocal}} to copy a file from HDFS to a destination
on an SMB share.  This is because in Hadoop 2, the {{copyToLocal}} maps to 2 underlying {{RawLocalFileSystem}}
operations: {{create}} and {{setPermission}}.  On an SMB share, the user may be authorized
for the {{create}} but denied for the {{setPermission}}.  Windows denies the {{WRITE_DAC}}
right required by {{setPermission}} unless the user has Full Control permissions.  Granting
Full Control isn't feasible for most deployments, because it's insecure.  This is a regression
from Hadoop 1, where {{copyToLocal}} only did a {{create}} and didn't do a separate {{setPermission}}.

This message was sent by Atlassian JIRA

View raw message