hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mingliang Liu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14443) Azure: Support retry and client side failover for authorization, SASKey and delegation token generation
Date Wed, 05 Jul 2017 23:26:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16075613#comment-16075613
] 

Mingliang Liu commented on HADOOP-14443:
----------------------------------------

I ran the tests for the branch-2 patch, and found the following errors.

{code}
Failed tests:
  TestWasbRemoteCallHelper.testMalFormedJSONResponse
Expected: (an instance of org.apache.hadoop.fs.azure.WasbAuthorizationException and exception
with message a string containing "com.fasterxml.jackson.core.JsonParseException: Unexpected
end-of-input in FIELD_NAME")
     but: an instance of org.apache.hadoop.fs.azure.WasbAuthorizationException <org.codehaus.jackson.JsonParseException:
Unexpected end-of-input within/between OBJECT entries
 at [Source: java.io.StringReader@6b3e12b5; line: 1, column: 131]> is a org.codehaus.jackson.JsonParseException
Stacktrace was: org.codehaus.jackson.JsonParseException: Unexpected end-of-input within/between
OBJECT entries
 at [Source: java.io.StringReader@6b3e12b5; line: 1, column: 131]
	at org.codehaus.jackson.JsonParser._constructError(JsonParser.java:1433)
	at org.codehaus.jackson.impl.ReaderBasedParser._skipWS(ReaderBasedParser.java:1470)
	at org.codehaus.jackson.impl.ReaderBasedParser.nextToken(ReaderBasedParser.java:425)
	at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:690)
	at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:580)
	at org.codehaus.jackson.map.ObjectReader._bindAndClose(ObjectReader.java:768)
	at org.codehaus.jackson.map.ObjectReader.readValue(ObjectReader.java:460)
	at org.apache.hadoop.fs.azure.RemoteWasbAuthorizerImpl.authorize(RemoteWasbAuthorizerImpl.java:153)
	at org.apache.hadoop.fs.azure.NativeAzureFileSystem.performAuthCheck(NativeAzureFileSystem.java:1468)
	at org.apache.hadoop.fs.azure.NativeAzureFileSystem.create(NativeAzureFileSystem.java:1704)
	at org.apache.hadoop.fs.azure.NativeAzureFileSystem.create(NativeAzureFileSystem.java:1554)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1067)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1048)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:937)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:925)
	at org.apache.hadoop.fs.azure.TestWasbRemoteCallHelper.performop(TestWasbRemoteCallHelper.java:445)
	at org.apache.hadoop.fs.azure.TestWasbRemoteCallHelper.testMalFormedJSONResponse(TestWasbRemoteCallHelper.java:247)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:168)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
{code}

Can you fix that on branch-2? Thanks,

> Azure: Support retry and client side failover for authorization, SASKey and delegation
token generation
> -------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-14443
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14443
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs/azure
>    Affects Versions: 2.9.0
>            Reporter: Santhosh G Nayak
>            Assignee: Santhosh G Nayak
>             Fix For: 3.0.0-alpha4
>
>         Attachments: HADOOP-14443.1.patch, HADOOP-14443.2.patch, HADOOP-14443.3.patch,
HADOOP-14443.4.patch, HADOOP-14443.5.patch, HADOOP-14443.6.patch, HADOOP-14443.7.patch, HADOOP-14443-branch2-1.patch
>
>
> Currently, {{WasRemoteCallHelper}} can be configured to talk to only one URL for authorization,
SASKey generation and delegation token generation. If for some reason the service is down,
all the requests will fail.
> So proposal is to,
> - Add support to configure multiple URLs, so that if communication to one URL fails,
client can retry on another instance of the service running on different node for authorization,
SASKey generation and delegation token generation. 
> - Rename the configurations {{fs.azure.authorization.remote.service.url}} to {{fs.azure.authorization.remote.service.urls}}
and {{fs.azure.cred.service.url}} to {{fs.azure.cred.service.urls}} to support the comma separated
list of URLs.
> - Introduce a new configuration {{fs.azure.delegation.token.service.urls}} to configure
the comma separated list of service URLs to get the delegation token.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message