flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Theofilos Kakantousis <t...@kth.se>
Subject Flink on Yarn delegation token renewal
Date Tue, 08 Nov 2016 09:25:50 GMT
Hi everyone,

I'm using Flink 1.1.3 with Hadoop 2.7.3 and was wondering about 
delegation token renewal when running Flink on Yarn. Yarn demands 
services to renew delegation tokens on their own and if not, Yarn jobs 
will fail after one week.
https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YarnApplicationSecurity.html

For example, Spark has implemented a solution for that
https://github.com/apache/spark/blob/v2.0.1/yarn/src/main/scala/org/apache/spark/deploy/yarn/AMDelegationTokenRenewer.scala

and people had reported issues running Flink in the past,
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-job-on-secure-Yarn-fails-after-many-hours-td3856.html

What is the status of this issue? How can Flink on Yarn handle this 
issue so that jobs do not fail due to token expiration?

Cheers,
Theo




Mime
View raw message