hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christophe Taton (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (HADOOP-1298) adding user info to file
Date Sat, 11 Aug 2007 18:27:43 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12519205
] 

kryzthov edited comment on HADOOP-1298 at 8/11/07 11:26 AM:
--------------------------------------------------------------------

Here is a quite complete description of the current state of this work so that you know what
is going here.

The patch relies on a subset of the authentication framework that should be provided in the
future by this issue [HADOOP-1701|https://issues.apache.org/jira/browse/HADOOP-1701]. It then
includes a subset of this framework for now and until HADOOP-1701 will be committed.
It provides:
* authorizations for most DFS File operations by extending the Java authorization framework
* a subset of POSIX permissions (no groups for now and no "execution" right)

To preserve compatibility, the default permission for all files is: owner=root and POSIX permissions
are rwx---rwx. All entities (web apps, fsck, clients...) for now use a fake Root user account.
The authorization framework should allow an easy transition toward a permission checked environment.

The patch is about 4000 lines (including HADOOP-1701 code). However it is still *incomplete*
(see the Todo list at the end of this message). All JUnit tests pass.

h3.Changes

* dfs.ClientProtocol:
*- added a Ticket parameter for most operations
*- added a setPermissions(Path, PermissionCollection) operation

* dfs.DFSClient:
*- added a Ticket parameter for most operations (symmetric of dfs.ClientProtocol)
*- dfs.DFSClient.DFSInputStream and DFSOutputStream: added a Ticket field (these are stateful
objects that keep the Ticket used to create them for further operations)

* dfs.DFSPolicy:
*- new file which implements a policy provider that relies on INode's permissions data to
generate policies; the current DFSPolicy concretely implements the POSIX permission checking
scheme for now (without groups)

* dfs.DFSSecurityManager:
*- new file which implements the SecurityManager for the JVM that runs the DFS
*- it currently ignores OS permissions checking as we trust Hadoop DFS code, and only processes
fs.permissions.HFilePermission
*- it also provides one helper method for each type of HFileAction we can check for

* dfs.DistributedFileSystem:
*- added a Ticket parameter implicitely used for all the operations provided by this client
interface
*- the Ticket is defaulted to a fake ROOT user for now so as to preserve compatibility, and
we will restrict this progressively after

* dfs.DSConstants:
*- updated the layout version

* dfs.FSDirectory:
*- added a WritablePermissionCollection field to INode; this field is defaulted to owner=root
and mode=rwx---rwx so as to preserve compatibility as much as possible and to allow easy and
progressive restrictions to be integrated in the future
*- added a getter and a setter for INode.permissions
*- added a INode getDeepestExistingNode(String path) method that copies the INode getNode(String
path) method but always returns an existing INode (the deepest one in the INode tree that
leads to the full given path); this to allow easy permission checking against the already
existing directory along the given path.

* dfs.FSImage:
*- updated the saveImage() method to write the permissions
*- updated the loadFSImage() method to read the permissions or provide a default permissions
(owner=root and mode=rwx---rwx) when upgrading from a previous layout

* dfs.FSNamesystem:
*- added a setPermissions() operation

* dfs.NameNode:
*- updated the init() method to setup the SecurityManager and the Policy provider for the
JVM
*- scattered DFSSecurityManager.checkDFS{Read,Write,Create,Delete,SetPermissions}() in all
operations
*- added a setPermissions() (to implement dfs.ClientProtocol)
*- added a few method to setup the appropriate Subject in the AccessControlContext (getSubjectOfMessage()
and doAsSubject())
*- wrapped all operation that have a Ticket parameter so as to extract the appropriate Subject
from the Ticket and run the real operation under this Subject's principal.

* dfs.NamenodeFsck:
*- updated to used a default Root Ticket

* dfs.StreamFile:
*- updated to use a default Root Ticket

* fs.permission: new package
*- fs.permission.HFileAction: represents the 5 actions related to DFS Files (create, delete,
read, write, set permissions)
*- fs.permission.HFilePermission: unmutable Permission object to represent a user requested
action on a file
*- fs.permission.POSIXFileAction: represents the 3 POSIX file actions (read, write, execute)
*- fs.permission.POSIXFilePermission: unmutable Permission object to represent a user requested
POSIX action on a file
*- fs.permission.POSIXFilePermissions: mutable WritablePermissionCollection that stores the
current permissions for an INode in terms of POSIX actions (contains an owner ID and a mode)
*- fs.permission.WritablePermissionCollection: abstract PermissionCollection that must implement
Writable also

* ipc.Client:
*- updated Writable call(Writable param) so that it can propagates AccessControlException
to the user process

* ipc.RPC:
*- updated Writable RPC.Server.call(Writable param) so that it correctly serializes AccessControlException

* security.simple.SimpleTicket:
*- updated to provide 3 fake users (for compatibility and for testing purpose)

* Tests
*- dfs.ClusterTestDFS, dfs.ClusterTestDFSNamespaceLogging, dfs.TestReplication: use the default
Root Ticket
*- dfs.TestPermission: new test case to test the underlying permission checking infrastructure
a bit

* Web apps
*- browseBlock.jsp, browseDirectory.jsp, tail.jsp: updated to use the default Root Ticket
(use DistributedFileSystem)


h3.To do

* High priority
*- Update the fsEditLog for the setPermissions() operation
*- Figure out how to provide a getPermissions() and to extend fs.FileStatus for clients
*- FsShell command to get/set permissions
*- Deep tests
*- real use of authentication

* Low priority
*- Sticky bit (really helpful to have a shared /scratch or /tmp directory where you don't
want others to remove your temporary files!)
*- Groups?


That's it for now!

I hope this will help you to get a better idea of how this has been designed for now.
Any comment will be highly appreciated.

Thanks,
Christophe T.

      was (Author: kryzthov):
    Here is a quite complete description of the current state of this work so that you know
what is going here.

The patch relies on a subset of the authentication framework that should be provided in the
future by this issue [HADOOP-1701|https://issues.apache.org/jira/browse/HADOOP-1701]. It then
includes a subset of this framework for now and until HADOOP-1701 will be committed.
It provides:
* authorizations for most DFS File operations by extending the Java authorization framework
* a subset of POSIX permissions (no groups for now and no "execution" right)

To preserve compatibility, the default permission for all files is: owner=root and POSIX permissions
are rwx---rwx. All entities (web apps, fsck, clients...) for now use a fake Root user account.
The authorization framework should allow an easy transition toward a permission checked environment.

The patch is about 4000 lines (including HADOOP-1701 code). However it is still *incomplete*
(see the Todo list at the end of this message). All JUnit tests pass.

h3.Changes

* dfs.ClientProtocol:
*- added a Ticket parameter for most operations
*- added a setPermissions(Path, PermissionCollection) operation

* dfs.DFSClient:
*- added a Ticket parameter for most operations (symmetric of dfs.ClientProtocol)
*- dfs.DFSClient.DFSInputStream and DFSOutputStream: added a Ticket field (these are stateful
objects that keep the Ticket used to create them for further operations)

* dfs.DFSPolicy:
*- new file which implements a policy provider that relies on INode's permissions data to
generate policies; the current DFSPolicy concretely implements the POSIX permission checking
scheme for now (without groups)

* dfs.DFSSecurityManager:
*- new file which implements the SecurityManager for the JVM that runs the DFS
*- it currently ignores OS permissions checking as we trust Hadoop DFS code, and only processes
fs.permissions.HFilePermission
*- it also provides one helper method for each type of HFileAction we can check for

* dfs.DistributedFileSystem:
*- added a Ticket parameter implicitely used for all the operations provided by this client
interface
*- the Ticket is defaulted to a fake ROOT user for now so as to preserve compatibility, and
we will restrict this progressively after

* dfs.DSConstants:
*- updated the layout version

* dfs.FSDirectory:
*- added a WritablePermissionCollection field to INode; this field is defaulted to owner=root
and mode=rwx---rwx so as to preserve compatibility as much as possible and to allow easy and
progressive restrictions to be integrated in the future
*- added a getter and a setter for INode.permissions
*- added a INode getDeepestExistingNode(String path) method that copies the INode getNode(String
path) method but always returns an existing INode (the deepest one in the INode tree that
leads to the full given path); this to allow easy permission checking against the already
existing directory along the given path.

* dfs.FSImage:
*- updated the saveImage() method to write the permissions
*- updated the loadFSImage() method to read the permissions or provide a default permissions
(owner=root and mode=rwx---rwx) when upgrading from a previous layout

* dfs.FSNamesystem:
*- added a setPermissions() operation

* dfs.NameNode:
*- updated the init() method to setup the SecurityManager and the Policy provider for the
JVM
*- scattered DFSSecurityManager.checkDFS{Read,Write,Create,Delete,SetPermissions}() in all
operations
*- added a setPermissions() (to implement dfs.ClientProtocol)
*- added a few method to setup the appropriate Subject in the AccessControlContext (getSubjectOfMessage()
and doAsSubject())
*- wrapped all operation that have a Ticket parameter so as to extract the appropriate Subject
from the Ticket and run the real operation under this Subject's principal.

* dfs.NamenodeFsck:
*- updated to used a default Root Ticket

* dfs.StreamFile:
*- updated to use a default Root Ticket

*fs.permission: new package
*- fs.permission.HFileAction: represents the 5 actions related to DFS Files (create, delete,
read, write, set permissions)
*- fs.permission.HFilePermission: unmutable Permission object to represent a user requested
action on a file
*- fs.permission.POSIXFileAction: represents the 3 POSIX file actions (read, write, execute)
*- fs.permission.POSIXFilePermission: unmutable Permission object to represent a user requested
POSIX action on a file
*- fs.permission.POSIXFilePermissions: mutable WritablePermissionCollection that stores the
current permissions for an INode in terms of POSIX actions (contains an owner ID and a mode)
*- fs.permission.WritablePermissionCollection: abstract PermissionCollection that must implement
Writable also

* ipc.Client:
*- updated Writable call(Writable param) so that it can propagates AccessControlException
to the user process

* ipc.RPC:
*- updated Writable RPC.Server.call(Writable param) so that it correctly serializes AccessControlException

* security.simple.SimpleTicket:
*- updated to provide 3 fake users (for compatibility and for testing purpose)

* Tests
*- dfs.ClusterTestDFS, dfs.ClusterTestDFSNamespaceLogging, dfs.TestReplication: use the default
Root Ticket
*- dfs.TestPermission: new test case to test the underlying permission checking infrastructure
a bit

* Web apps
*- browseBlock.jsp, browseDirectory.jsp, tail.jsp: updated to use the default Root Ticket
(use DistributedFileSystem)


h3.To do

* High priority
*- Update the fsEditLog for the setPermissions() operation
*- Figure out how to provide a getPermissions() and to extend fs.FileStatus for clients
*- FsShell command to get/set permissions
*- Deep tests
*- real use of authentication

* Low priority
*- Sticky bit (really helpful to have a shared /scratch or /tmp directory where you don't
want others to remove your temporary files!)
*- Groups?


That's it for now!

I hope this will help you to get a better idea of how this has been designed for now.
Any comment will be highly appreciated.

Thanks,
Christophe T.
  
> adding user info to file
> ------------------------
>
>                 Key: HADOOP-1298
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1298
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: dfs, fs
>            Reporter: Kurtis Heimerl
>             Fix For: 0.15.0
>
>         Attachments: hadoop-authorization-20070810b.patch, hadoop-user-munncha.patch,
hadoop-user-munncha.patch, hadoop-user-munncha.patch, hadoop-user-munncha.patch10, hadoop-user-munncha.patch11,
hadoop-user-munncha.patch12, hadoop-user-munncha.patch13, hadoop-user-munncha.patch14, hadoop-user-munncha.patch15,
hadoop-user-munncha.patch16, hadoop-user-munncha.patch17, hadoop-user-munncha.patch4, hadoop-user-munncha.patch5,
hadoop-user-munncha.patch6, hadoop-user-munncha.patch7, hadoop-user-munncha.patch8, hadoop-user-munncha.patch9,
layout20070731.patch
>
>
> I'm working on adding a permissions model to hadoop's DFS. The first step is this change,
which associates user info with files. Following this I'll assoicate permissions info, then
block methods based on that user info, then authorization of the user info. 
> So, right now i've implemented adding user info to files. I'm looking for feedback before
I clean this up and make it offical. 
> I wasn't sure what release, i'm working off trunk. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message