hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mike Drob (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15570) renewable delegation tokens for long-lived spark applications
Date Fri, 26 May 2017 23:05:05 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16027001#comment-16027001

Mike Drob commented on HBASE-15570:

[~busbey] - I think SPARK-12523 is the one that fixes this, not SPARK-14743, available in
Spark 2.0

I'm not sure that a fix to the Spark 1.6.z line will be coming.

> renewable delegation tokens for long-lived spark applications
> -------------------------------------------------------------
>                 Key: HBASE-15570
>                 URL: https://issues.apache.org/jira/browse/HBASE-15570
>             Project: HBase
>          Issue Type: Improvement
>          Components: spark
>            Reporter: Sean Busbey
>            Assignee: Sean Busbey
> Right now our spark integration works on secure clusters by getting delegation tokens
and sending them to the executors. Unfortunately, applications that need to run for longer
than the delegation token lifetime (by default 7 days) will fail.
> In particular, this is an issue for Spark Streaming applications. Since they expect to
run indefinitely, we should have a means for renewing the delegation tokens.

This message was sent by Atlassian JIRA

View raw message