hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-11321) copyToLocal cannot save a file to an SMB share unless the user has Full Control permissions.
Date Tue, 09 Dec 2014 23:32:14 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-11321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Chris Nauroth updated HADOOP-11321:
    Attachment: HADOOP-11321.003.patch

Patch v3 implements a native code solution for creating local file system directories and
files with an initial security descriptor equivalent to the mode passed by the caller.  I
have tested this locally.  It fixes the original reported problem for a Windows client accessing
an SMB share.  For a Linux client accessing an SMB share, there is no change in behavior in
the Hadoop code, and it still works fine via the error handling in Linux CIFS mounts (silently
ignoring the chmod failure).

Here is a description of the changes.
* {{RawLocalFileSystem}}: On Windows, make a JNI call to open a file descriptor for write
instead of using {{java.io.FileOutputStream}}.  There are 2 new methods that can be overridden
in subclasses: {{mkOneDirWithMode}} and {{createOutputStreamWithMode}}.  These are intended
to support {{ElevatedRawLocalFileSystem}}, discussed in more detail below.  We cannot remove
the older protected methods {{mkOneDir}} and {{createOutputStream}}, because the class is
annotated {{Public}} and {{Stable}}.  These are now implemented as simple forwarding methods.
* {{ChecksumFileSystem}}: This needed to change some of the file creation calls it uses on
the wrapped file system to pass down the mode.  There is one awkward point here that required
an {{instanceof}} check, but I think it's preferable over adding more abstract methods to
{{FileSystem}} just to cover one special case.  Several calls are rerouted to equivalent overloaded
methods that can accept mode as an argument.
* {{NativeIO}}: Added Windows-specific JNI wrappers for a native implementation of directory
and file creation.  For file creation, I needed to be careful to match the semantics that
were implemented by {{java.io.FileOutputStream}}.  This involved reading a bit of the JDK
code, and I've described the details in comments.
* {{NativeIO.c}}: Added JNI implementations for the 2 new functions.  These are pretty boring
and just pass through to the real action in...
* {{libwinutils.c}}: This contains the implementation of calling the relevant Windows APIs.
 I've reused our implementation of translating a mode to a discretionary access control list
({{GetWindowsDACLs}}), so this will be consistent with, for example, {{winutils.exe chmod}}.
* {{WindowsSecureContainerExecutor}}: This needed some adjustment to be able to handle the
changes required in {{RawLocalFileSystem}}.  [~rusanu], this implementation still does creation
and setting permissions in separate syscalls.  I suppose we could consolidate to a single
syscall if we update the IDL for the RPC protocol to accept a mode argument, but I didn't
want to tackle that in this patch.  I also don't know the compatibility constraints around
versioning this RPC protocol.  This change was focused on a bug fix related to SMB shares.
 I don't believe this is a major limitation for {{ElevatedRawLocalFileSystem}}.  If NodeManager
is running backed by an SMB share for the localization directory, then there are probably
lots of other problems anyway.  :-)  If you think it's important to come back to this, please
file a separate jira for that.

> copyToLocal cannot save a file to an SMB share unless the user has Full Control permissions.
> --------------------------------------------------------------------------------------------
>                 Key: HADOOP-11321
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11321
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 2.6.0
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
>         Attachments: HADOOP-11321.003.patch, HADOOP-11321.1.patch, HADOOP-11321.2.patch,
> In Hadoop 2, it is impossible to use {{copyToLocal}} to copy a file from HDFS to a destination
on an SMB share.  This is because in Hadoop 2, the {{copyToLocal}} maps to 2 underlying {{RawLocalFileSystem}}
operations: {{create}} and {{setPermission}}.  On an SMB share, the user may be authorized
for the {{create}} but denied for the {{setPermission}}.  Windows denies the {{WRITE_DAC}}
right required by {{setPermission}} unless the user has Full Control permissions.  Granting
Full Control isn't feasible for most deployments, because it's insecure.  This is a regression
from Hadoop 1, where {{copyToLocal}} only did a {{create}} and didn't do a separate {{setPermission}}.

This message was sent by Atlassian JIRA

View raw message