hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Seth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5663) Add an interface to Input/Ouput Formats to obtain delegation tokens
Date Wed, 11 Dec 2013 19:47:07 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13845676#comment-13845676

Siddharth Seth commented on MAPREDUCE-5663:

Proposed API / Changes

interface CredentialsProvider (in hadoop-common)
   void addCredentials(Configuration conf, Credentials credentials);

Change at least FileInput/OutputFormats to implement this interface to return the tokens that
will eventually be required. We can additional Input/OutputFormats as required.

Any system (MR/Tez) which wants to do AM split generation - would need to check if the interface
is implemented by the relevant Input/OutputFormat - and decide whether to generate splits
on the client side or just get tokens on the client side when running on a secure cluster.

> Add an interface to Input/Ouput Formats to obtain delegation tokens
> -------------------------------------------------------------------
>                 Key: MAPREDUCE-5663
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5663
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Siddharth Seth
>            Assignee: Siddharth Seth
> Currently, delegation tokens are obtained as part of the getSplits / checkOutputSpecs
calls to the InputFormat / OutputFormat respectively.
> This works as long as the splits are generated on a node with kerberos credentials. For
split generation elsewhere (AM for example), an explicit interface is required.

This message was sent by Atlassian JIRA

View raw message