hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Urko Benito (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-506) Incorrect UserName at Solaris because it has no "whoami" command by default
Date Wed, 29 Jul 2009 12:34:15 GMT

    [ https://issues.apache.org/jira/browse/HDFS-506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12736609#action_12736609
] 

Urko Benito commented on HDFS-506:
----------------------------------

There're some issues and i'll try to explain what's happend onto Solaris.


1.- By default the Shell class calls to "whoami" without full path, this is a security risk


Here's the example

*1.- Create a new directory onto Hadoop HDFS on Solaris 10  and set the _user_ and _group_
to *hadoop*, then set the perm to _700_ so, only *hadoop* user can _read/write_ onto*

$ uname -a
SunOS sol10-enigma 5.10 Generic_139556-08 i86pc i386 i86pc
$ hadoop fs -mkdir testdir
$  hadoop fs -chown -R hadoop:hadoop testdir
$ hadoop fs -chmod 700 testdir
$ hadoop fs -lsr
drwx------   - hadoop hadoop          0 2009-07-29 13:43 /user/hadoop/testdir


*2.- Now from _another_ Sun Box try to run the sample application that i've attached ( test-hadoop-security.tar.gz
) this one has no whoami at path*

$ uname -a
SunOS hedwig 5.10 Generic_127127-11 sun4u sparc SUNW,Sun-Fire-V490
$ id
uid=2011(postgres) gid=90(postgres)
$ /usr/xpg4/bin/id -Gn
postgres dba
$ /usr/xpg4/bin/id -un
postgres
$ which whoami
no whoami in /opt/apache-ant-1.7.1/bin /u01/app/postgres/8.4/db/bin/64 /u01/app/postgres/8.4/db/bin
/usr/bin /usr/sfw/bin /usr/local/bin

*3.- Run the Sample Application*

$ cd TestHadoop/
$ ant
Buildfile: build.xml

init:
    [mkdir] Created dir: /export/home/postgres/TestHadoop/build
    [mkdir] Created dir: /export/home/postgres/TestHadoop/javadoc
    [mkdir] Created dir: /export/home/postgres/TestHadoop/build.test

compile:
    [javac] Compiling 1 source file to /export/home/postgres/TestHadoop/build
    [javac] /export/home/postgres/TestHadoop/src/com/sfchildren/sbcore/backup/Main.java:89:
warning: [deprecation] USER_NAME_COMMAND in org.apache.hadoop.util.Shell has been deprecated
    [javac]              String[] result = executeShellCommand( new String[]{Shell.USER_NAME_COMMAND});
    [javac]                                                                       ^
    [javac] 1 warning

dist:
    [mkdir] Created dir: /export/home/postgres/TestHadoop/dist
      [jar] Building jar: /export/home/postgres/TestHadoop/dist/hadoop-example.jar

run:
     [echo]
     [echo]             Test Hadoop Shell.USER_NAME without "whois" at path
     [echo]
     [echo]             1.- Test with original Hadoop-0.20.0-core.jar
     [echo]             2.- Test with *NEW* Hadoop-0.20.1-core-dev.jar
     [echo]
     [echo]             TestCase
     [echo]              - Create 3 directories "uno" "dos" "tres" at "/user/hadoop/testdir"
     [echo]             the "/user/hadoop/testdir"  has the been created as bellow
     [echo]
     [echo]             $ hadoop fs -mkdir testdir
     [echo]             $ hadoop fs -chown -R hadoop:hadoop testdir
     [echo]             $ hadoop fs -chmod 700 testdir
     [echo]             $ hadoop fs -lsr
     [echo]             drwx------   - hadoop hadoop          0 2009-07-29 13:24 /user/hadoop/testdir
     [echo]
     [echo]             - Create 1 file onto directory
     [echo]
     [echo]
     [echo]
     [echo]
     [echo]
     [echo]             Running with hadoop-0.20.0-core.jar
     [echo]
     [echo]
     [java] Get UserName from System.getProperty: postgres
     [java] Get UserName from Shell.USER_NAME_COMMAND Fail, Default User DrWho assigned
     [java] 29-jul-2009 14:16:59 com.sfchildren.sbcore.backup.Main createTmpFile
     [java] INFO: File testfile Created with Size: 772889 bytes Time: 1 seconds
     [java] 29-jul-2009 14:17:04 com.sfchildren.sbcore.backup.Main createTmpFile
     [java] INFO: File testfile Created with Size: 3592563 bytes Time: 4 seconds
     [java] 29-jul-2009 14:17:20 com.sfchildren.sbcore.backup.Main createTmpFile
     [java] INFO: File testfile Created with Size: 12417028 bytes Time: 16 seconds
     [echo]
     [echo]
     [echo]             Running With New Hadoop-0.20.1-dev-core
     [echo]
     [echo]
     [java] Get UserName from System.getProperty: postgres
     [java] Get UserName from Shell.USER_NAME_COMMAND Fail, Default User DrWho assigned
     [java] Get UserName from Shell.getUSER_NAME_COMMAND() postgres
     [java] Exception org.apache.hadoop.security.AccessControlException: Permission denied:
user=postgres, access=EXECUTE, inode="testdir":hadoop:hadoop:rwx------

BUILD SUCCESSFUL
Total time: 27 seconds

as you can see, with the *hadoop-0.20-core.jar* the user is ignored, but with me *hadoop-0.20.1-dev-core.jar*
the _user_ and _group_ is correct identified as _postgres_ and group _postgres_ and a _AccessControlException_
is launched when try to access _testdir_


*4.- Verify the HDFS structure now*

$ hadoop fs -lsr
drwx------   - hadoop hadoop          0 2009-07-29 13:50 /user/hadoop/testdir
drwxr-xr-x   - DrWho hadoop          0 2009-07-29 13:50 /user/hadoop/testdir/dos
-rw-r--r--   3 DrWho hadoop    3592563 2009-07-29 13:50 /user/hadoop/testdir/dos/testfile
drwxr-xr-x   - DrWho hadoop          0 2009-07-29 13:50 /user/hadoop/testdir/tres
-rw-r--r--   3 DrWho hadoop   12417028 2009-07-29 13:50 /user/hadoop/testdir/tres/testfile
drwxr-xr-x   - DrWho hadoop          0 2009-07-29 13:50 /user/hadoop/testdir/uno
-rw-r--r--   3 DrWho hadoop     772889 2009-07-29 13:50 /user/hadoop/testdir/uno/testfile


*5.- The hdfs-site.xml i've defined the value dfs.permission.supergroup to _hadoop_ as*
{quote}
    <!-- SuperUser Group -->
    <property>
        <name>dfs.permissions.supergroup</name>
        <value>hadoop</value>
        <final>true</final>
    </property>
{quote}

But the user who launched the TestApp is _postgres_ with groups _postgres and dba_ no *hadoop*



The Second issue is if i make a "fake" of _whois_ i can _bypass_ the security layout as the
above example


*1.- At Hadoop Cluster clean and create the _testdir_*
$ hadoop fs -rmr testdir
Deleted hdfs://namenode.hadoop.test.com:54310/user/hadoop/testdir
$ hadoop fs -mkdir testdir
$  hadoop fs -chown -R hadoop:hadoop testdir
$ hadoop fs -chmod 700 testdir
$ hadoop fs -lsr
drwx------   - hadoop hadoop          0 2009-07-29 14:02 /user/hadoop/testdir

*2.- At the second SunBox try to make a _whoami fake_*

$ uname -a
SunOS hedwig 5.10 Generic_127127-11 sun4u sparc SUNW,Sun-Fire-V490
$ which whoami
no whoami in /opt/apache-ant-1.7.1/bin /u01/app/postgres/8.4/db/bin/64 /u01/app/postgres/8.4/db/bin
/usr/bin /usr/sfw/bin /usr/local/bin
$ echo "echo hadoop" > ~/whoami
$ chmod +x ~/whoami
$ export PATH=~:$PATH
$ which whoami
/export/home/postgres/whoami
$ whoami
hadoop
$ id
uid=2011(postgres) gid=90(postgres)

*3.- Launch TestApp*

$ cd TestHadoop/
$ ant
Buildfile: build.xml

init:

compile:

dist:

run:
     [echo]
     [echo]             Test Hadoop Shell.USER_NAME without "whois" at path
     [echo]
     [echo]             1.- Test with original Hadoop-0.20.0-core.jar
     [echo]             2.- Test with *NEW* Hadoop-0.20.1-core-dev.jar
     [echo]
     [echo]             TestCase
     [echo]              - Create 3 directories "uno" "dos" "tres" at "/user/hadoop/testdir"
     [echo]             the "/user/hadoop/testdir"  has the been created as bellow
     [echo]
     [echo]             $ hadoop fs -mkdir testdir
     [echo]             $ hadoop fs -chown -R hadoop:hadoop testdir
     [echo]             $ hadoop fs -chmod 700 testdir
     [echo]             $ hadoop fs -lsr
     [echo]             drwx------   - hadoop hadoop          0 2009-07-29 13:24 /user/hadoop/testdir
     [echo]
     [echo]             - Create 1 file onto directory
     [echo]
     [echo]
     [echo]
     [echo]
     [echo]
     [echo]             Running with hadoop-0.20.0-core.jar
     [echo]
     [echo]
     [java] Get UserName from System.getProperty: postgres
     [java] Get UserName from Shell.USER_NAME_COMMAND hadoop
     [java] 29-jul-2009 14:32:45 com.sfchildren.sbcore.backup.Main createTmpFile
     [java] INFO: File testfile Created with Size: 4421872 bytes Time: 4 seconds
     [java] 29-jul-2009 14:32:53 com.sfchildren.sbcore.backup.Main createTmpFile
     [java] INFO: File testfile Created with Size: 7235384 bytes Time: 8 seconds
     [java] 29-jul-2009 14:33:02 com.sfchildren.sbcore.backup.Main createTmpFile
     [java] INFO: File testfile Created with Size: 5693610 bytes Time: 8 seconds
     [echo]
     [echo]
     [echo]             Running With New Hadoop-0.20.1-dev-core
     [echo]
     [echo]
     [java] Get UserName from System.getProperty: postgres
     [java] Get UserName from Shell.USER_NAME_COMMAND hadoop
     [java] Get UserName from Shell.getUSER_NAME_COMMAND() postgres
     [java] Exception org.apache.hadoop.security.AccessControlException: Permission denied:
user=postgres, access=EXECUTE, inode="testdir":hadoop:hadoop:rwx------

BUILD SUCCESSFUL
Total time: 24 seconds


As you can see, now hadoop say that _user_ is _hadoop_ 

*4.- View HDFS files*

$ hadoop fs -lsr
drwx------   - hadoop hadoop          0 2009-07-29 14:02 /user/hadoop/testdir
$ hadoop fs -lsr
drwx------   - hadoop hadoop          0 2009-07-29 14:06 /user/hadoop/testdir
drwxr-xr-x   - hadoop hadoop          0 2009-07-29 14:06 /user/hadoop/testdir/dos
-rw-r--r--   3 hadoop hadoop    7235384 2009-07-29 14:06 /user/hadoop/testdir/dos/testfile
drwxr-xr-x   - hadoop hadoop          0 2009-07-29 14:06 /user/hadoop/testdir/tres
-rw-r--r--   3 hadoop hadoop    5693610 2009-07-29 14:06 /user/hadoop/testdir/tres/testfile
drwxr-xr-x   - hadoop hadoop          0 2009-07-29 14:06 /user/hadoop/testdir/uno
-rw-r--r--   3 hadoop hadoop    4421872 2009-07-29 14:06 /user/hadoop/testdir/uno/testfile




I think now it's more clear no?

I've attached some files to evaluate the new code. 

> Incorrect UserName at Solaris because it has no "whoami" command by default
> ---------------------------------------------------------------------------
>
>                 Key: HDFS-506
>                 URL: https://issues.apache.org/jira/browse/HDFS-506
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: build
>    Affects Versions: 0.20.1
>         Environment: OS: SunOS 5.10
>            Reporter: Urko Benito
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Solaris enviroment has no __whoami__ command, so the __getUnixUserName()__ at UnixUserGroupInformation
class fails because it's calling to Shell.USER_NAME_COMMAND which is defines as "whoami".
> So it launched an Exception and set the default "DrWho" username ignoring all the FileSystem
permissions.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message