hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edward.y...@samsung.com>
Subject RE: Error occurred when using ContainerManagementProtocol
Date Fri, 30 Jan 2015 00:30:06 GMT
Hi,

I'm not familiar with YARN but it looks like you need to check whether
Application Master has NMToken[1] or not. And set the LOG level to DEBUG. 

I hope to see Hama on Yarn soon, Good Luck, Kim!

1.
http://hadoop.apache.org/docs/r2.4.0/api/org/apache/hadoop/yarn/api/records/
NMToken.html

--
Best Regards, Edward J. Yoon


-----Original Message-----
From: 김민호 [mailto:minwise.kim@samsung.com] 
Sent: Thursday, January 29, 2015 5:48 PM
To: yarn-dev@hadoop.apache.org
Cc: dev@hama.apache.org
Subject: Error occurred when using ContainerManagementProtocol

Hi,

 

I have been refactoring Hama yarn module on HDP 2.1(Hadoop 2.4.0).

However, the error occurred in the code where Application Master runs the
container with ContainerManagementProtocol object.

 

This is the log of error as follows.

FATAL bsp.BSPApplicationMaster: Error starting BSPApplicationMaster

java.io.IOException: Failed on local exception: java.io.IOException:
org.apache.hadoop.security.AccessControlException: Client cannot
authenticate via:[TOKEN]; Host Details : local host is: "slave1.hama.com";
destination host is: "slave1.hama.com":45454;

    at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:764)

    at org.apache.hadoop.ipc.Client.call(Client.java:1414)

    at org.apache.hadoop.ipc.Client.call(Client.java:1363)

    at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.jav
a:206)

    at com.sun.proxy.$Proxy17.startContainers(Unknown Source)

    at
org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagementProtocolPBClien
tImpl.startContainers(ContainerManagementProtocolPBClientImpl.java:96)

    at
org.apache.hama.bsp.BSPTaskLauncher.setupContainer(BSPTaskLauncher.java:222)

    at org.apache.hama.bsp.BSPTaskLauncher.start(BSPTaskLauncher.java:94)

    at org.apache.hama.bsp.JobImpl.startJob(JobImpl.java:240)

    at
org.apache.hama.bsp.BSPApplicationMaster.start(BSPApplicationMaster.java:304
)

    at
org.apache.hama.bsp.BSPApplicationMaster.main(BSPApplicationMaster.java:350)

Caused by: java.io.IOException:
org.apache.hadoop.security.AccessControlException: Client cannot
authenticate via:[TOKEN]

    at org.apache.hadoop.ipc.Client$Connection$1.run(Client.java:677)

    at java.security.AccessController.doPrivileged(Native Method)

    at javax.security.auth.Subject.doAs(Subject.java:415)

    at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.ja
va:1594)

    at
org.apache.hadoop.ipc.Client$Connection.handleSaslConnectionFailure(Client.j
ava:640)

    at
org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:724)

    at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:367)

    at org.apache.hadoop.ipc.Client.getConnection(Client.java:1462)

    at org.apache.hadoop.ipc.Client.call(Client.java:1381)

    ... 9 more

Caused by: org.apache.hadoop.security.AccessControlException: Client cannot
authenticate via:[TOKEN]

    at
org.apache.hadoop.security.SaslRpcClient.selectSaslClient(SaslRpcClient.java
:172)

    at
org.apache.hadoop.security.SaslRpcClient.saslConnect(SaslRpcClient.java:394)

    at
org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client.java:550)

    at org.apache.hadoop.ipc.Client$Connection.access$1800(Client.java:367)

    at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:716)

    at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:712)

    at java.security.AccessController.doPrivileged(Native Method)

    at javax.security.auth.Subject.doAs(Subject.java:415)

    at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.ja
va:1594)

    at
org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:711)

    ... 12 more

 

I tried to use following code which gets a token in order to use
ContainerManagementProtocol.

 

NMTokenSecretManagerInRM nmTokenSecretManagerInRM =

          new NMTokenSecretManagerInRM(conf);

String user = conf.get("bsp.user.name");

if (user == null) {

user = System.getenv(ApplicationConstants.Environment.USER.name());

}

Token nmToken = nmTokenSecretManagerInRM

          .createNMToken(appAttemptId, allocatedContainer.getNodeId(),
user);

 

But nmTokenSecretManagerInRM is null.

 

FATAL bsp.BSPApplicationMaster: Error starting BSPApplicationMaster

java.lang.NullPointerException

    at
org.apache.hadoop.yarn.server.security.BaseNMTokenSecretManager.createNMToke
n(BaseNMTokenSecretManager.java:128)

    at org.apache.hama.bsp.JobImpl.startJob(JobImpl.java:219)

    at
org.apache.hama.bsp.BSPApplicationMaster.start(BSPApplicationMaster.java:304
)

    at
org.apache.hama.bsp.BSPApplicationMaster.main(BSPApplicationMaster.java:350)

 

How can I get a token to use ContainerManagementProtocol?

 

Regards,

Minho Kim



Mime
View raw message