hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prabhu Joseph <prabhujose.ga...@gmail.com>
Subject Re: Long running Spark job on YARN throws "No AMRMToken"
Date Tue, 09 Feb 2016 05:55:57 GMT
+ Spark-Dev

On Tue, Feb 9, 2016 at 10:04 AM, Prabhu Joseph <prabhujose.gates@gmail.com>
wrote:

> Hi All,
>
>     A long running Spark job on YARN throws below exception after running
> for few days.
>
> yarn.ApplicationMaster: Reporter thread fails 1 time(s) in a row.
> org.apache.hadoop.yarn.exceptions.YarnException: *No AMRMToken found* for
> user prabhu at org.apache.hadoop.yarn.ipc.RPC
> Util.getRemoteException(RPCUtil.java:45)
>
> Do any of the below renew the AMRMToken and solve the issue
>
> 1. yarn-resourcemanager.delegation.token.max-lifetime increase from 7 days
>
> 2. Configuring Proxy user:
>
> <property> <name>hadoop.proxyuser.yarn.hosts</name> <value>*</value>
> </property>
> <property> <name>hadoop.proxyuser.yarn.groups</name> <value>*</value>
> </property>
>
> 3. Can Spark-1.4.0 handle with fix
> https://issues.apache.org/jira/browse/SPARK-5342
>
>     spark.yarn.credentials.file
>
>
> How to renew the AMRMToken for a long running job on YARN?
>
>
> Thanks,
> Prabhu Joseph
>
>
>
>
>

Mime
View raw message