hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergey Murylev <sergeymury...@gmail.com>
Subject Re: UserGroupInformation: PriviledgedActionException as:root (auth:SIMPLE)
Date Sat, 12 Apr 2014 17:11:23 GMT
Hi,

> 1 - I don't understand why this is happening. I should have been able
> to copy data. Why I can't copy data between hdfs? 
I think you should check permissions not only for root ("/"), you need
to make sure that whole path is accessible for root. You can do this
using following command:
> /root/Programs/hadoop/bin/hadoop fs -cat
> webhdfs://WorkGroup0000:50070/wiki/wiki/wv92.txt 
Make sure that folders /wiki and /wiki/wiki can be listed by root.
Actually I've confused by your use case, I don't understand why you use
webhdfs instead of hdfs. Native hdfs protocol is more faster than webhdfs.
> 2 - Is it possible to turn off the privilege rules? 
Yes, it is possible, you should add flowing configuration to
core-site.xml and restart mapreduce & hdfs daemons
>   <property>
>     <name>dfs.permissions</name>
>     <value>false</value>
>   </property>

--
Thanks,
Sergey

On 12/04/14 20:52, xeon wrote:
> Hi,
>
> I want to copy data between hdfs in different hosts using webhdfs in
> hadoop MRv2. The destination host has the user and group as root and
> supergroup.
>
> WorkGroup0010:~/# hdfs dfs -ls /
> 14/04/12 16:42:18 WARN util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes
> where applicable
> Found 3 items
> drwxrwx---   - root supergroup          0 2014-04-12 16:25 /root
> drwxr-xr-x   - root supergroup          0 2014-04-12 16:25 /tmp
> drwxrwxrwx   - root supergroup          0 2014-04-12 16:39 /wiki
>
> I defined in the core-site.xml these params for the user and the group.
>
> WorkGroup0010:~/# cat ../hadoop/etc/hadoop/core-site.xml
> <?xml version="1.0" encoding="UTF-8"?>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>
> <!-- Put site-specific property overrides in this file. -->
> <configuration>
>   <property> <name>fs.default.name</name>
> <value>hdfs://172.16.100.5:9000</value> </property>
>   <property> <name>hadoop.tmp.dir</name>
> <value>/tmp/hadoop-temp</value> </property>
> <property><name>hadoop.proxyuser.root.hosts</name><value>*</value></property>
>
> <property><name>hadoop.proxyuser.supergroup.groups</name><value>*</value></property>
> <-- I am trying to use with thisgroup definition
>   <!--
> property><name>hadoop.proxyuser.root.groups</name><value>*</value></property
> --> <-- or with this group
> </configuration>
>
> When I copy data
>
> WorkGroup0000:~/# /root/Programs/hadoop/bin/hadoop distcp
> webhdfs://WorkGroup0000:50070/wiki webhdfs://WorkGroup0010:50070/wiki
>
> I get the error on the destination as:
> 2014-04-12 16:37:06,720 ERROR
> org.apache.hadoop.security.UserGroupInformation:
> PriviledgedActionException as:root (auth:SIMPLE)
> cause:java.io.FileNotFoundException: File does not exist:
> /wiki/wiki/wv92.txt
>
> 1 - I don't understand why this is happening. I should have been able
> to copy data. Why I can't copy data between hdfs?
> 2 - Is it possible to turn off the privilege rules?
>


Mime
View raw message