hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From REYANE OUKPEDJO <r.oukpe...@yahoo.com>
Subject Yarn Localization failure on hadoop
Date Mon, 27 Apr 2015 18:44:21 GMT
Hi there,
I have  an Application that is trying to launch an AM , but the localization is failing with
the below error message. The Resource visiblity is set to private which means the localizer
will run through a container executor as a user that submit the job. I checked that hdfs dfs
-ls / command is working fine as a user sumbitting the Application to make sure kerberos credentials
for this user is not to be blamed.  a mapreduce  pi example could run successfully. Any
idea what could cause this kind of issue?

NODE MANAGER LOGS HERE 2015-04-27 22:30:48,682 INFO  authorize.ServiceAuthorizationManager
(ServiceAuthorizationManager.java:authorize(114)) - Authorization successful for testing (auth:TOKEN)
for protocol=interface org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB2015-04-27
22:30:49,463 INFO  localizer.ResourceLocalizationService (ResourceLocalizationService.java:update(932))
- DEBUG: FAILED { hdfs://datanode5.in.ibm.com:8020/user/dsadm/.staging/AppMaster.jar, 1430154129414,
FILE, null }, Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException:
Client cannot authenticate via:[TOKEN, KERBEROS]; Host Details : local host is: "datanode3.in.ibm.com/";
destination host is: "datanode5.in.ibm.com":8020;2015-04-27 22:30:49,463 INFO  localizer.LocalizedResource
(LocalizedResource.java:handle(196)) - Resource hdfs://datanode5.in.ibm.com:8020/user/dsadm/.staging/AppMaster.jar
transitioned from DOWNLOADING to FAILED2015-04-27 22:30:49,463 INFO  container.Container
(ContainerImpl.java:handle(901)) - Container container_1429925144518_0016_01_000001 transitioned
from LOCALIZING to LOCALIZATION_FAILED2015-04-27 22:30:49,464 INFO  localizer.LocalResourcesTrackerImpl
(LocalResourcesTrackerImpl.java:handle(137)) - Container container_1429925144518_0016_01_000001
sent RELEASE event on a resource request { hdfs://datanode5.in.ibm.com:8020/user/dsadm/.staging/AppMaster.jar,
1430154129414, FILE, null } not present in cache.2015-04-27 22:30:49,464 WARN  nodemanager.NMAuditLogger
(NMAuditLogger.java:logFailure(150)) - USER=dsadm       OPERATION=Container Finished -
Failed   TARGET=ContainerImpl    RESULT=FAILURE  DESCRIPTION=Container failed with state:
LOCALIZATION_FAILED    APPID=application_1429925144518_0016    CONTAINERID=container_1429925144518_0016_01_0000012015-04-27
22:30:49,464 INFO  container.Container (ContainerImpl.java:handle(901)) - Container container_1429925144518_0016_01_000001
transitioned from LOCALIZATION_FAILED to DONE2015-04-27 22:30:49,465 INFO  application.Application
(ApplicationImpl.java:transition(339)) - Removing container_1429925144518_0016_01_000001 from
application application_1429925144518_00162015-04-27 22:30:49,465 INFO  monitor.ContainersMonitorImpl
(ContainersMonitorImpl.java:isEnabled(169)) - Neither virutal-memory nor physical-memory monitoring
is needed. Not running the monitor-thread2015-04-27 22:30:49,465 INFO  containermanager.AuxServices
(AuxServices.java:handle(175)) - Got event CONTAINER_STOP for appId application_1429925144518_0016@

View raw message