Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Date: Sat, 15 Sep 2012 12:04:07 +1100 (NCT)
From: "Bikas Saha (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <1154196738.83372.1347671047825.JavaMail.jiratomcat@arcas>
In-Reply-To: <556339847.2085.1346088967901.JavaMail.jiratomcat@arcas>
Subject: [jira] [Commented] (HADOOP-8731) Public distributed cache support
 for Windows
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-8731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13456270#comment-13456270 ] 

Bikas Saha commented on HADOOP-8731:
------------------------------------

Ivan, I think Vinod means the following.
In a real distributed cluster where filesystem is HDFS, then the current methods work because dist cache files are on hdfs and hdfs permissions resemble posix. So isPublic() etc is called on HDFS filesystem. So things will work.
When running using the local file system, say in the LocalJobRunner scenario, dist cache files are on local fs. In that case the current methods like isPublic() do not work on Windows because of the reasons who mentioned. This is what is happening in the test. 

                
> Public distributed cache support for Windows
> --------------------------------------------
>
>                 Key: HADOOP-8731
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8731
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: filecache
>            Reporter: Ivan Mitic
>            Assignee: Ivan Mitic
>         Attachments: HADOOP-8731-PublicCache.patch
>
>
> A distributed cache file is considered public (sharable between MR jobs) if OTHER has read permissions on the file and +x permissions all the way up in the folder hierarchy. By default, Windows permissions are mapped to "700" all the way up to the drive letter, and it is unreasonable to ask users to change the permission on the whole drive to make the file public. IOW, it is hardly possible to have public distributed cache on Windows. 
> To enable the scenario and make it more "Windows friendly", the criteria on when a file is considered public should be relaxed. One proposal is to check whether the user has given EVERYONE group permission on the file only (and discard the +x check on parent folders).
> Security considerations for the proposal: Default permissions on Unix platforms are usually "775" or "755" meaning that OTHER users can read and list folders by default. What this also means is that Hadoop users have to explicitly make the files private in order to make them private in the cluster (please correct me if this is not the case in real life!). On Windows, default permissions are "700". This means that by default all files are private. In the new model, if users want to make them public, they have to explicitly add EVERYONE group permissions on the file. 
> TestTrackerDistributedCacheManager fails because of this issue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira