hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Lu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6565) Configuration to use host name in delegation token service is not read from job.xml during MapReduce job execution.
Date Thu, 24 Nov 2016 00:06:58 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15691724#comment-15691724
] 

Li Lu commented on MAPREDUCE-6565:
----------------------------------

That's not exactly the same issue as we observed here. Can you try latest trunk to verify
this issue? I would not be surprised if problems occur on all YARN apps after a fix in YarnClient,
but I believe the issue discussed here is solely about MR. 

> Configuration to use host name in delegation token service is not read from job.xml during
MapReduce job execution.
> -------------------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6565
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6565
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Chris Nauroth
>            Assignee: Li Lu
>         Attachments: MAPREDUCE-6565-trunk.001.patch
>
>
> By default, the service field of a delegation token is populated based on server IP address.
 Setting {{hadoop.security.token.service.use_ip}} to {{false}} changes this behavior to use
host name instead of IP address.  However, this configuration property is not read from job.xml.
 Instead, it's read from a separate {{Configuration}} instance created during static initialization
of {{SecurityUtil}}.  This does not work correctly with MapReduce jobs if the framework is
distributed by setting {{mapreduce.application.framework.path}} and the {{mapreduce.application.classpath}}
is isolated to avoid reading core-site.xml from the cluster nodes.  MapReduce tasks will fail
to authenticate to HDFS, because they'll try to find a delegation token based on the NameNode
IP address, even though at job submission time the tokens were generated using the host name.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message