hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mingliang Liu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14443) Azure: Add retry and client side failover for authorization, SASKey generation and delegation token generation requests to remote service
Date Wed, 31 May 2017 23:56:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16032227#comment-16032227
] 

Mingliang Liu commented on HADOOP-14443:
----------------------------------------

I will review this one shortly.

Early minor comments:
# Can you make the @Override annotation in a separate line?
# The URI build code like following, can we make sure the parameters such as {{op}}, {{renewer}}
and {{service}} constant?
{code}
106	  @Override public Token<DelegationTokenIdentifier> getDelegationToken(
107	      String renewer) throws IOException {
108	    URIBuilder uriBuilder =
109	        new URIBuilder().setPath(DEFAULT_DELEGATION_TOKEN_MANAGER_ENDPOINT)
110	            .addParameter("op", GET_DELEGATION_TOKEN_OP)
111	            .addParameter("renewer", renewer)
112	            .addParameter("service", WASB_DT_SERVICE_NAME.toString());
113	    String responseBody = remoteCallHelper
114	        .makeRemoteRequest(dtServiceUrls, uriBuilder.getPath(),
115	            uriBuilder.getQueryParams(), HttpGet.METHOD_NAME);
116	    return TokenUtils.toDelegationToken(JsonUtils.parse(responseBody));
117	  }
{code}
# Do we need configurable retry policy? I'm not sure about this.
# In {{WasbRemoteCallHelper::shouldRetry()}}, I think we may need to self-interrupt and return
in case of InterruptedException for the {{Thread.sleep(a.delayMillis)}}. Currently we throw
{{WasbRemoteCallException}} which may be too strict.
# This is a large patch. I'd appreciate if you can separate the code refactoring work for
tokens from the retry logic. That will speed up the review and commit progress.

Thanks,

> Azure: Add retry and client side failover for authorization, SASKey generation and delegation
token generation requests to remote service
> -----------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-14443
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14443
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs/azure
>    Affects Versions: 2.9.0
>            Reporter: Santhosh G Nayak
>            Assignee: Santhosh G Nayak
>             Fix For: 2.9.0, 3.0.0-alpha4
>
>         Attachments: HADOOP-14443.1.patch, HADOOP-14443.2.patch
>
>
> Currently, {{WasRemoteCallHelper}} can be configured to talk to only one URL for authorization,
SASKey generation and delegation token generation. If for some reason the service is down,
all the requests will fail.
> So proposal is to,
> - Add support to configure multiple URLs, so that if communication to one URL fails,
client can retry on another instance of the service running on different node for authorization,
SASKey generation and delegation token generation. 
> - Rename the configurations {{fs.azure.authorization.remote.service.url}} to {{fs.azure.authorization.remote.service.urls}}
and {{fs.azure.cred.service.url}} to {{fs.azure.cred.service.urls}} to support the comma separated
list of URLs.
> Introduce a new configuration {{fs.azure.delegation.token.service.urls}} to configure
the comma separated list of service URLs to get the delegation token.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message