hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kartashov, Andy" <Andy.Kartas...@mpac.ca>
Subject RE: Datanode: "Cannot start secure cluster without privileged resources"
Date Mon, 26 Nov 2012 14:52:00 GMT
Try running  $sudo jps or jps as root "#jps"

You will get more info, i.e:

****Jps
***** SecondaryNameNode
***** JobTracker
***** NameNode


-----Original Message-----
From: ac@hsk.hk [mailto:ac@hsk.hk]
Sent: Monday, November 26, 2012 9:44 AM
To: Harsh J
Cc: ac@hsk.hk; <user@hadoop.apache.org>
Subject: Re: Datanode: "Cannot start secure cluster without privileged resources"

Hi,

I think you are right!



1) $ jps
16152
16500 Jps


2) ps axu | grep 16152
hduser   16152  0.1  1.4 1834900 116760 ?      Sl   21:34   0:06 jsvc.exec -Dproc_datanode
-outfile /usr/local/hadoop-1.0.4/libexec/../logs/jsvc.out -errfile /usr/local/hadoop-1.0.4/libexec/../logs/jsvc.err
-pidfile /tmp/hadoop_secure_dn.pid -nodetach -user hduser -cp /usr/local/hadoop-1.0.4/libexec/../conf:/usr/lib/jvm/lib/tools.jar:/usr/local/hadoop-1.0.4/libexec/..:/usr/local/hadoop-1.0.4/libexec/../hadoop-core-1.0.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/asm-3.2.jar:/usr/local/hadoop-1.0.4/libexec/../lib/aspectjrt-1.6.5.jar:/usr/local/hadoop-1.0.4/libexec/../lib/aspectjtools-1.6.5.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-beanutils-1.7.0.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-beanutils-core-1.8.0.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-cli-1.2.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-codec-1.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-collections-3.2.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-configuration-1.6.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-daemon-1.0.10.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-digester-1.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-el-1.0.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-httpclient-3.0.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-io-2.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-lang-2.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-logging-1.1.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-logging-api-1.0.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-math-2.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-net-1.4.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/core-3.1.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/hadoop-capacity-scheduler-1.0.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/hadoop-fairscheduler-1.0.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/hadoop-thriftfs-1.0.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/hsqldb-1.8.0.10.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jackson-core-asl-1.8.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jackson-mapper-asl-1.8.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jasper-compiler-5.5.12.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jasper-runtime-5.5.12.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jdeb-0.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jersey-core-1.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jersey-json-1.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jersey-server-1.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jets3t-0.6.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jetty-6.1.26.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jetty-util-6.1.26.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jsch-0.1.42.jar:/usr/local/hadoop-1.0.4/libexec/../lib/junit-4.5.jar:/usr/local/hadoop-1.0.4/libexec/../lib/kfs-0.2.2.jar:/usr/local/hadoop-1.0.4/libexec/../lib/log4j-1.2.15.jar:/usr/local/hadoop-1.0.4/libexec/../lib/mockito-all-1.8.5.jar:/usr/local/hadoop-1.0.4/libexec/../lib/oro-2.0.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/servlet-api-2.5-20081211.jar:/usr/local/hadoop-1.0.4/libexec/../lib/slf4j-api-1.4.3.jar:/usr/local/hadoop-1.0.4/libexec/../lib/slf4j-log4j12-1.4.3.jar:/usr/local/hadoop-1.0.4/libexec/../lib/xmlenc-0.52.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jsp-2.1/jsp-2.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jsp-2.1/jsp-api-2.1.jar
-Xmx1000m -jvm server -Xmx1024m -Dsecurity.audit.logger=ERROR,DRFAS -Dcom.sun.management.jmxremote
-Xmx1024m -Dsecurity.audit.logger=ERROR,DRFAS -Dcom.sun.management.jmxremote -Xmx1024m -Dsecurity.audit.logger=ERROR,DRFAS
-Dcom.sun.management.jmxremote -Dhadoop.log.dir=/usr/local/hadoop-1.0.4/libexec/../logs -Dhadoop.log.file=hadoop-hduser-datanode-m147.log
-Dhadoop.home.dir=/usr/local/hadoop-1.0.4/libexec/.. -Dhadoop.id.str=hduser -Dhadoop.root.logger=INFO,DRFA
-Dhadoop.security.logger=INFO,NullAppender -Djava.library.path=/usr/local/hadoop-1.0.4/libexec/../lib/native/Linux-amd64-64
-Dhadoop.policy.file=hadoop-policy.xml org.apache.hadoop.hdfs.server.datanode.SecureDataNodeStarter
root     16497  0.0  0.0   9384   924 pts/0    S+   22:35   0:00 grep --color=auto 16152


3) ps axu | grep 16117
root     16117  0.0  0.0  17004   904 ?        S    21:34   0:00 jsvc.exec -Dproc_datanode
-outfile /usr/local/hadoop-1.0.4/libexec/../logs/jsvc.out -errfile /usr/local/hadoop-1.0.4/libexec/../logs/jsvc.err
-pidfile /tmp/hadoop_secure_dn.pid -nodetach -user hduser -cp /usr/local/hadoop-1.0.4/libexec/../conf:/usr/lib/jvm/lib/tools.jar:/usr/local/hadoop-1.0.4/libexec/..:/usr/local/hadoop-1.0.4/libexec/../hadoop-core-1.0.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/asm-3.2.jar:/usr/local/hadoop-1.0.4/libexec/../lib/aspectjrt-1.6.5.jar:/usr/local/hadoop-1.0.4/libexec/../lib/aspectjtools-1.6.5.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-beanutils-1.7.0.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-beanutils-core-1.8.0.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-cli-1.2.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-codec-1.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-collections-3.2.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-configuration-1.6.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-daemon-1.0.10.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-digester-1.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-el-1.0.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-httpclient-3.0.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-io-2.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-lang-2.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-logging-1.1.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-logging-api-1.0.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-math-2.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/commons-net-1.4.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/core-3.1.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/hadoop-capacity-scheduler-1.0.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/hadoop-fairscheduler-1.0.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/hadoop-thriftfs-1.0.4.jar:/usr/local/hadoop-1.0.4/libexec/../lib/hsqldb-1.8.0.10.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jackson-core-asl-1.8.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jackson-mapper-asl-1.8.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jasper-compiler-5.5.12.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jasper-runtime-5.5.12.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jdeb-0.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jersey-core-1.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jersey-json-1.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jersey-server-1.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jets3t-0.6.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jetty-6.1.26.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jetty-util-6.1.26.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jsch-0.1.42.jar:/usr/local/hadoop-1.0.4/libexec/../lib/junit-4.5.jar:/usr/local/hadoop-1.0.4/libexec/../lib/kfs-0.2.2.jar:/usr/local/hadoop-1.0.4/libexec/../lib/log4j-1.2.15.jar:/usr/local/hadoop-1.0.4/libexec/../lib/mockito-all-1.8.5.jar:/usr/local/hadoop-1.0.4/libexec/../lib/oro-2.0.8.jar:/usr/local/hadoop-1.0.4/libexec/../lib/servlet-api-2.5-20081211.jar:/usr/local/hadoop-1.0.4/libexec/../lib/slf4j-api-1.4.3.jar:/usr/local/hadoop-1.0.4/libexec/../lib/slf4j-log4j12-1.4.3.jar:/usr/local/hadoop-1.0.4/libexec/../lib/xmlenc-0.52.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jsp-2.1/jsp-2.1.jar:/usr/local/hadoop-1.0.4/libexec/../lib/jsp-2.1/jsp-api-2.1.jar
-Xmx1000m -jvm server -Xmx1024m -Dsecurity.audit.logger=ERROR,DRFAS -Dcom.sun.management.jmxremote
-Xmx1024m -Dsecurity.audit.logger=ERROR,DRFAS -Dcom.sun.management.jmxremote -Xmx1024m -Dsecurity.audit.logger=ERROR,DRFAS
-Dcom.sun.management.jmxremote -Dhadoop.log.dir=/usr/local/hadoop-1.0.4/libexec/../logs -Dhadoop.log.file=hadoop-hduser-datanode-m147.log
-Dhadoop.home.dir=/usr/local/hadoop-1.0.4/libexec/.. -Dhadoop.id.str=hduser -Dhadoop.root.logger=INFO,DRFA
-Dhadoop.security.logger=INFO,NullAppender -Djava.library.path=/usr/local/hadoop-1.0.4/libexec/../lib/native/Linux-amd64-64
-Dhadoop.policy.file=hadoop-policy.xml org.apache.hadoop.hdfs.server.datanode.SecureDataNodeStarter
root     16499  0.0  0.0   9388   920 pts/0    R+   22:35   0:00 grep --color=auto 16117



I started all DNs in secure mode now.
Thanks again!

ac

On 26 Nov 2012, at 10:30 PM, Harsh J wrote:

> Could you also check what 16152 is? The jsvc is a launcher process,
> not the JVM itself.
>
> As I mentioned, JPS is pretty reliable, just wont' show the name of
> the JVM launched by a custom wrapper - and will show just PID.
>
> On Mon, Nov 26, 2012 at 7:35 PM, ac@hsk.hk <ac@hsk.hk> wrote:
>> Hi,
>>
>> Thanks for your reply.
>>
>> However, I think 16152 should not be the DN, since
>> 1) my second try of "/usr/local/hadoop/bin/hadoop-daemon.sh start datanode" says
16117 (i.e. I ran start datanode twice), and
>> 2) ps axu | grep 16117, I got
>> root     16117  0.0  0.0  17004   904 pts/2    S    21:34   0:00 jsvc.exec -Dproc_datanode
-outfile /usr/local/hadoop-1.0.4/libexec/ ...
>>
>> These are the two reasons that I think JPS is no longer a tool to check secure DN.
>>
>> Thanks again!
>>
>>
>> On 26 Nov 2012, at 9:47 PM, Harsh J wrote:
>>
>>> The 16152 should be the DN JVM I think. This is a jps limitation, as
>>> seen at http://docs.oracle.com/javase/1.5.0/docs/tooldocs/share/jps.html
>>> and jsvc (which secure mode DN uses) is such a custom launcher.
>>>
>>> "The jps command uses the java launcher to find the class name and
>>> arguments passed to the main method. If the target JVM is started with
>>> a custom launcher, the class name (or JAR file name) and the arguments
>>> to the main method will not be available. In this case, the jps
>>> command will output the string Unknownfor the class name or JAR file
>>> name and for the arguments to the main method."
>>>
>>> On Mon, Nov 26, 2012 at 7:11 PM, ac@hsk.hk <ac@hsk.hk> wrote:
>>>> Hi,
>>>>
>>>> A question:
>>>> I started Secure DN then ran JPS as root, I could not find any running DN:
>>>> 16152
>>>> 16195 Jps
>>>>
>>>> However, when I tried to start the secure DN again, I got:
>>>> Warning: $HADOOP_HOME is deprecated.
>>>> datanode running as process 16117. Stop it first.
>>>>
>>>> Does it mean JPS is no longer a tool to check DN in secure mode?
>>>>
>>>> Thanks
>>>>
>>>>
>>>> On 26 Nov 2012, at 9:03 PM, ac@hsk.hk wrote:
>>>>
>>>>> Hi Harsh,
>>>>>
>>>>> Thank you very much for your reply, got it!
>>>>>
>>>>> Thanks
>>>>> ac
>>>>>
>>>>> On 26 Nov 2012, at 8:32 PM, Harsh J wrote:
>>>>>
>>>>>> Secure DN needs to be started as root (it runs as proper user, but
>>>>>> needs to be started as root to grab reserved ports), and needs a
>>>>>> proper jsvc binary (for your arch/OS) available. Are you using
>>>>>> tarballs or packages (and if packages, are they from Bigtop)?
>>>>>>
>>>>>> On Mon, Nov 26, 2012 at 5:21 PM, ac@hsk.hk <ac@hsk.hk> wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> I am setting up HDFS security with Kerberos:
>>>>>>> When I manually started the first datanode, I got the following
messages (the namenode is started):
>>>>>>>
>>>>>>> 1) INFO org.apache.hadoop.security.UserGroupInformation: Login
successful for user ....
>>>>>>> 2) ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.lang.RuntimeException:
Cannot start secure cluster without privileged resources.
>>>>>>>
>>>>>>> OS: Ubuntu 12.04
>>>>>>> Hadoop: 1.0.4
>>>>>>>
>>>>>>> It seems that it could login successfully but something is missing
>>>>>>> Please help!
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Harsh J
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Harsh J
>>
>
>
>
> --
> Harsh J

NOTICE: This e-mail message and any attachments are confidential, subject to copyright and
may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not
the intended recipient, please delete and contact the sender immediately. Please consider
the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe
qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts
par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite.
Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement
l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel
Mime
View raw message