hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Seth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5663) Add an interface to Input/Ouput Formats to obtain delegation tokens
Date Sat, 28 Dec 2013 00:03:51 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13857850#comment-13857850
] 

Siddharth Seth commented on MAPREDUCE-5663:
-------------------------------------------

Thanks for taking this up [~michaelweng]. , some comments on the patch.
- FileInputFormat - a new public getInputPaths(...) method should not be added. This could
instead be a private getInputPathsInternal(...)
- FileOutputFormat - path normalization, similar to what is done, in checkOutputSpecs is likely
required.
- CredentialsProvider - the JavaDoc should be a little more descriptive as to the purpose
of this interface. Was thinking along the lines of 
{code}
/**
   * <code>CredentialsProvider</code> is an interface that can be implemented
by
   * components that may need to obtain credentials, which may
   * be required to function on a secured cluster.
   * 
   * @param conf
   *          component specific {@link Configuration}.
   * @param credentials
   *          an instance of {@link Credentials} to which credentials
   *          will be added.
   *
/{code}
Could also do with some changes to the FIF/FOF unit tests.

> Add an interface to Input/Ouput Formats to obtain delegation tokens
> -------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5663
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5663
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Siddharth Seth
>            Assignee: Michael Weng
>         Attachments: MAPREDUCE-5663.patch.txt, MAPREDUCE-5663.patch.txt2, MAPREDUCE-5663.patch.txt3
>
>
> Currently, delegation tokens are obtained as part of the getSplits / checkOutputSpecs
calls to the InputFormat / OutputFormat respectively.
> This works as long as the splits are generated on a node with kerberos credentials. For
split generation elsewhere (AM for example), an explicit interface is required.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message