hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chen Liang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-14035) NN status discovery does not leverage delegation token
Date Wed, 31 Oct 2018 17:57:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-14035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670465#comment-16670465
] 

Chen Liang commented on HDFS-14035:
-----------------------------------

Had some discussion offline. Turns out one issue with this approach is that it will associate
HDFS DelegationToken with HAServiceProtocol ALL the time. Say, in the future if we have another
use case using HAServiceProtocol but has nothing to do with HDFS (HAServiceProtocol is meant
to be generic after all), it will be tricky to prevent returning an out-of-nowhere HDFS delegation
token selector in that case. Actually, this rises the high level question of whether HAServiceProtocol
should be using HDFS delegation token at all.

In the long term, a better approach might be to introduce a new protocol such as HAStatusProtocol
which just separates getServiceStatus functionality out of HAServiceProtocol and start from
there. But that is not in the scope of this feature branch and should probably be discussed
separately with more folks from trunk. 

Additionally, in our case, getServiceStatus returns a HAServiceStatus instance, but HAServiceStatus
has more than what is actually needed, what we really need is just HAServiceStatus#HAServiceState.

So for now, I propose to just add the functionality of server state discover to ClientNamenodeProtocol
and replace using HAServiceProtocol here. Specifically, introduce a new RPC call such as getServerHAState
in ClientNamenodeProtocol, which just returns something like a HAServiceState instance. If
we are to separate getServiceStatus out of HAServiceProtocol, we can revisit and refactor
in future. Will repurpose this JIRA.

cc [~shv], [~xkrogen], [~csun]

> NN status discovery does not leverage delegation token
> ------------------------------------------------------
>
>                 Key: HDFS-14035
>                 URL: https://issues.apache.org/jira/browse/HDFS-14035
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Chen Liang
>            Assignee: Chen Liang
>            Priority: Major
>         Attachments: HDFS-14035-HDFS-12943.001.patch, HDFS-14035-HDFS-12943.002.patch
>
>
> Currently ObserverReadProxyProvider uses {{HAServiceProtocol#getServiceStatus}} to get
the status of each NN. However {{HAServiceProtocol}} does not leverage delegation token. So
when running an application on YARN and when YARN node manager makes this call getServiceStatus, token
authentication will fail, causing the application to fail.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message