hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick Dimiduk (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15570) renewable delegation tokens for long-lived spark applications
Date Thu, 31 Mar 2016 15:38:25 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15220050#comment-15220050

Nick Dimiduk commented on HBASE-15570:

Is this specific to the Spark integration? Does it apply to OutputFormat implementations as
well -- ie, Spark/Flink Streaming writing through OutputFormat.

> renewable delegation tokens for long-lived spark applications
> -------------------------------------------------------------
>                 Key: HBASE-15570
>                 URL: https://issues.apache.org/jira/browse/HBASE-15570
>             Project: HBase
>          Issue Type: Improvement
>          Components: spark
>            Reporter: Sean Busbey
>            Assignee: Sean Busbey
> Right now our spark integration works on secure clusters by getting delegation tokens
and sending them to the executors. Unfortunately, applications that need to run for longer
than the delegation token lifetime (by default 7 days) will fail.
> In particular, this is an issue for Spark Streaming applications. Since they expect to
run indefinitely, we should have a means for renewing the delegation tokens.

This message was sent by Atlassian JIRA

View raw message