hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ac@hsk.hk" ...@hsk.hk>
Subject Re: Datanode: "Cannot start secure cluster without privileged resources"
Date Mon, 26 Nov 2012 14:43:51 GMT
Hi,

I think you are right!



1) $ jps
16152 
16500 Jps


2) ps axu | grep 16152
hduser   16152  0.1  1.4 1834900 116760 ?      Sl   21:34   0:06 jsvc.exec -Dproc_datanode
-outfile /usr/local/hadoop-1.0.4/libexec/../logs/jsvc.out -errfile /usr/local/hadoop-1.0.4/libexec/../logs/jsvc.err
-pidfile /tmp/hadoop_secure_dn.pid -nodetach -user hduser -cp /usr/local/hadoop-1.0.4/libexec/../conf:/usr/lib/jvm/lib/tools.jar:/usr/local/hadoop-1.0.4/libexec/..:/usr/local/hadoop-1.0.4/libexec/../hadoop-core-1.0.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/asm-3.2.jar:/usr/local/hadoop-1.0.4/libexec/../lib/aspectjrt-1.6.5.jar:/usr/local/hadoop-1.0.4/libexec/../lib/aspectjtools-1.6.5.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-beanutils-1.7.0.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-beanutils-core-1.8.0.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-cli-1.2.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-codec-1.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-collections-3.2.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-configuration-1.6.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-daemon-1.0.10.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-digester-1.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-el-1.0.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-httpclient-3.0.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-io-2.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-lang-2.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-logging-1.1.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-logging-api-1.0.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-math-2.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-net-1.4.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/core-3.1.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/hadoop-capacity-scheduler-1.0.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/hadoop-fairscheduler-1.0.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/hadoop-thriftfs-1.0.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/hsqldb-1.8.0.10.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jackson-core-asl-1.8.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jackson-mapper-asl-1.8.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jasper-compiler-5.5.12.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jasper-runtime-5.5.12.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jdeb-0.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jersey-core-1.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jersey-json-1.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jersey-server-1.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jets3t-0.6.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jetty-6.1.26.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jetty-util-6.1.26.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jsch-0.1.42.jar:/usr/local/hadoop-1.0.4/libexec/../lib/junit-4.5.jar:/usr/local/hadoop-1.0.4/libexec/../lib/kfs-0.2.2.jar:/usr/local/hadoop-1.0.4/libexec/../lib/log4j-1.2.15.jar:/usr/local/hadoop-1.0.4/libexec/../lib/mockito-all-1.8.5.jar:/usr/local/hadoop-1.0.4/libexec/../lib/oro-2.0.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/servlet-api-2.5-20081211.jar:/usr/local/hadoop-1.0.4/libexec/../lib/slf4j-api-1.4.3.jar:/usr/local/hadoop-1.0.4/libexec/../lib/slf4j-log4j12-1.4.3.jar:/usr/local/hadoop-1.0.4/libexec/../lib/xmlenc-0.52.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jsp-2.1/jsp-2.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jsp-2.1/jsp-api-2.1.jar
-Xmx1000m -jvm server -Xmx1024m -Dsecurity.audit.logger=ERROR,DRFAS -Dcom.sun.management.jmxremote
-Xmx1024m -Dsecurity.audit.logger=ERROR,DRFAS -Dcom.sun.management.jmxremote -Xmx1024m -Dsecurity.audit.logger=ERROR,DRFAS
-Dcom.sun.management.jmxremote -Dhadoop.log.dir=/usr/local/hadoop-1.0.4/libexec/../logs -Dhadoop.log.file=hadoop-hduser-datanode-m147.log
-Dhadoop.home.dir=/usr/local/hadoop-1.0.4/libexec/.. -Dhadoop.id.str=hduser -Dhadoop.root.logger=INFO,DRFA
-Dhadoop.security.logger=INFO,NullAppender -Djava.library.path=/usr/local/hadoop-1.0.4/libexec/../lib/native/Linux-amd64-64
-Dhadoop.policy.file=hadoop-policy.xml org.apache.hadoop.hdfs.server.datanode.SecureDataNodeStarter
root     16497  0.0  0.0   9384   924 pts/0    S+   22:35   0:00 grep --color=auto 16152


3) ps axu | grep 16117
root     16117  0.0  0.0  17004   904 ?        S    21:34   0:00 jsvc.exec -Dproc_datanode
-outfile /usr/local/hadoop-1.0.4/libexec/../logs/jsvc.out -errfile /usr/local/hadoop-1.0.4/libexec/../logs/jsvc.err
-pidfile /tmp/hadoop_secure_dn.pid -nodetach -user hduser -cp /usr/local/hadoop-1.0.4/libexec/../conf:/usr/lib/jvm/lib/tools.jar:/usr/local/hadoop-1.0.4/libexec/..:/usr/local/hadoop-1.0.4/libexec/../hadoop-core-1.0.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/asm-3.2.jar:/usr/local/hadoop-1.0.4/libexec/../lib/aspectjrt-1.6.5.jar:/usr/local/hadoop-1.0.4/libexec/../lib/aspectjtools-1.6.5.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-beanutils-1.7.0.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-beanutils-core-1.8.0.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-cli-1.2.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-codec-1.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-collections-3.2.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-configuration-1.6.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-daemon-1.0.10.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-digester-1.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-el-1.0.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-httpclient-3.0.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-io-2.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-lang-2.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-logging-1.1.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-logging-api-1.0.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-math-2.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-net-1.4.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/core-3.1.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/hadoop-capacity-scheduler-1.0.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/hadoop-fairscheduler-1.0.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/hadoop-thriftfs-1.0.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/hsqldb-1.8.0.10.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jackson-core-asl-1.8.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jackson-mapper-asl-1.8.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jasper-compiler-5.5.12.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jasper-runtime-5.5.12.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jdeb-0.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jersey-core-1.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jersey-json-1.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jersey-server-1.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jets3t-0.6.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jetty-6.1.26.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jetty-util-6.1.26.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jsch-0.1.42.jar:/usr/local/hadoop-1.0.4/libexec/../lib/junit-4.5.jar:/usr/local/hadoop-1.0.4/libexec/../lib/kfs-0.2.2.jar:/usr/local/hadoop-1.0.4/libexec/../lib/log4j-1.2.15.jar:/usr/local/hadoop-1.0.4/libexec/../lib/mockito-all-1.8.5.jar:/usr/local/hadoop-1.0.4/libexec/../lib/oro-2.0.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/servlet-api-2.5-20081211.jar:/usr/local/hadoop-1.0.4/libexec/../lib/slf4j-api-1.4.3.jar:/usr/local/hadoop-1.0.4/libexec/../lib/slf4j-log4j12-1.4.3.jar:/usr/local/hadoop-1.0.4/libexec/../lib/xmlenc-0.52.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jsp-2.1/jsp-2.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jsp-2.1/jsp-api-2.1.jar
-Xmx1000m -jvm server -Xmx1024m -Dsecurity.audit.logger=ERROR,DRFAS -Dcom.sun.management.jmxremote
-Xmx1024m -Dsecurity.audit.logger=ERROR,DRFAS -Dcom.sun.management.jmxremote -Xmx1024m -Dsecurity.audit.logger=ERROR,DRFAS
-Dcom.sun.management.jmxremote -Dhadoop.log.dir=/usr/local/hadoop-1.0.4/libexec/../logs -Dhadoop.log.file=hadoop-hduser-datanode-m147.log
-Dhadoop.home.dir=/usr/local/hadoop-1.0.4/libexec/.. -Dhadoop.id.str=hduser -Dhadoop.root.logger=INFO,DRFA
-Dhadoop.security.logger=INFO,NullAppender -Djava.library.path=/usr/local/hadoop-1.0.4/libexec/../lib/native/Linux-amd64-64
-Dhadoop.policy.file=hadoop-policy.xml org.apache.hadoop.hdfs.server.datanode.SecureDataNodeStarter
root     16499  0.0  0.0   9388   920 pts/0    R+   22:35   0:00 grep --color=auto 16117



I started all DNs in secure mode now.
Thanks again!

ac

On 26 Nov 2012, at 10:30 PM, Harsh J wrote:

> Could you also check what 16152 is? The jsvc is a launcher process,
> not the JVM itself.
> 
> As I mentioned, JPS is pretty reliable, just wont' show the name of
> the JVM launched by a custom wrapper - and will show just PID.
> 
> On Mon, Nov 26, 2012 at 7:35 PM, ac@hsk.hk <ac@hsk.hk> wrote:
>> Hi,
>> 
>> Thanks for your reply.
>> 
>> However, I think 16152 should not be the DN, since
>> 1) my second try of "/usr/local/hadoop/bin/hadoop-daemon.sh start datanode" says
16117 (i.e. I ran start datanode twice), and
>> 2) ps axu | grep 16117, I got
>> root     16117  0.0  0.0  17004   904 pts/2    S    21:34   0:00 jsvc.exec -Dproc_datanode
-outfile /usr/local/hadoop-1.0.4/libexec/ ...
>> 
>> These are the two reasons that I think JPS is no longer a tool to check secure DN.
>> 
>> Thanks again!
>> 
>> 
>> On 26 Nov 2012, at 9:47 PM, Harsh J wrote:
>> 
>>> The 16152 should be the DN JVM I think. This is a jps limitation, as
>>> seen at http://docs.oracle.com/javase/1.5.0/docs/tooldocs/share/jps.html
>>> and jsvc (which secure mode DN uses) is such a custom launcher.
>>> 
>>> "The jps command uses the java launcher to find the class name and
>>> arguments passed to the main method. If the target JVM is started with
>>> a custom launcher, the class name (or JAR file name) and the arguments
>>> to the main method will not be available. In this case, the jps
>>> command will output the string Unknownfor the class name or JAR file
>>> name and for the arguments to the main method."
>>> 
>>> On Mon, Nov 26, 2012 at 7:11 PM, ac@hsk.hk <ac@hsk.hk> wrote:
>>>> Hi,
>>>> 
>>>> A question:
>>>> I started Secure DN then ran JPS as root, I could not find any running DN:
>>>> 16152
>>>> 16195 Jps
>>>> 
>>>> However, when I tried to start the secure DN again, I got:
>>>> Warning: $HADOOP_HOME is deprecated.
>>>> datanode running as process 16117. Stop it first.
>>>> 
>>>> Does it mean JPS is no longer a tool to check DN in secure mode?
>>>> 
>>>> Thanks
>>>> 
>>>> 
>>>> On 26 Nov 2012, at 9:03 PM, ac@hsk.hk wrote:
>>>> 
>>>>> Hi Harsh,
>>>>> 
>>>>> Thank you very much for your reply, got it!
>>>>> 
>>>>> Thanks
>>>>> ac
>>>>> 
>>>>> On 26 Nov 2012, at 8:32 PM, Harsh J wrote:
>>>>> 
>>>>>> Secure DN needs to be started as root (it runs as proper user, but
>>>>>> needs to be started as root to grab reserved ports), and needs a
>>>>>> proper jsvc binary (for your arch/OS) available. Are you using
>>>>>> tarballs or packages (and if packages, are they from Bigtop)?
>>>>>> 
>>>>>> On Mon, Nov 26, 2012 at 5:21 PM, ac@hsk.hk <ac@hsk.hk> wrote:
>>>>>>> Hi,
>>>>>>> 
>>>>>>> I am setting up HDFS security with Kerberos:
>>>>>>> When I manually started the first datanode, I got the following
messages (the namenode is started):
>>>>>>> 
>>>>>>> 1) INFO org.apache.hadoop.security.UserGroupInformation: Login
successful for user ....
>>>>>>> 2) ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.lang.RuntimeException:
Cannot start secure cluster without privileged resources.
>>>>>>> 
>>>>>>> OS: Ubuntu 12.04
>>>>>>> Hadoop: 1.0.4
>>>>>>> 
>>>>>>> It seems that it could login successfully but something is missing
>>>>>>> Please help!
>>>>>>> 
>>>>>>> Thanks
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Harsh J
>>>>> 
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> Harsh J
>> 
> 
> 
> 
> -- 
> Harsh J


Mime
View raw message