hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mingliang Liu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-14461) Azure: handle failure gracefully in case of missing account access key
Date Tue, 06 Jun 2017 22:46:18 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-14461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Mingliang Liu updated HADOOP-14461:
-----------------------------------
    Attachment: HADOOP-14461.002.patch

While the account access key is missing,

{code:title=before}
$ hadoop fs -ls wasb://test@liuml07.blob.core.windows.net/
ls: org.apache.hadoop.fs.azure.AzureException: Container test in account liuml07.blob.core.windows.net
not found, and we can't create it using anoynomous credentials, and no credentials found for
them in the configuration.
{code}

{code:title=after}
$ hadoop fs -ls wasb://test@liuml07.blob.core.windows.net/
ls: org.apache.hadoop.fs.azure.AzureException: No credentials found for account liuml07.blob.core.windows.net
in the configuration, and we are unable to access container test in this account using anonymous
credentials. Please check if the container exists first. If it is not publicly available,
you have to provide account credentials
{code}

The v2 patch has been tested against US WEST region. All tests pass except {{TestFileSystemOperationExceptionHandling}}
and {{TestFileSystemOperationsExceptionHandlingMultiThreaded}} which are not related here.
They are caused by [HADOOP-14478] and are being tracked in [HADOOP-14500].
{code:title=after reverting HADOOP-14478}
hadoop/hadoop-tools/hadoop-azure $ mvn test -q
Results :

Tests run: 703, Failures: 0, Errors: 0, Skipped: 119
{code}

Ping [~snayak] and [~stevel@apache.org] for review.

> Azure: handle failure gracefully in case of missing account access key
> ----------------------------------------------------------------------
>
>                 Key: HADOOP-14461
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14461
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/azure
>            Reporter: Mingliang Liu
>            Assignee: Mingliang Liu
>         Attachments: HADOOP-14461.000.patch, HADOOP-14461.001.patch, HADOOP-14461.002.patch
>
>
> Currently if the {{fs.azure.account.key.youraccount}} is missing, we will get error stack
like this:
> {code}
> java.lang.IllegalArgumentException: The String is not a valid Base64-encoded string.
> 	at com.microsoft.azure.storage.core.Base64.decode(Base64.java:63)
> 	at com.microsoft.azure.storage.StorageCredentialsAccountAndKey.<init>(StorageCredentialsAccountAndKey.java:81)
> 	at org.apache.hadoop.fs.azure.AzureBlobStorageTestAccount.createStorageAccount(AzureBlobStorageTestAccount.java:464)
> 	at org.apache.hadoop.fs.azure.AzureBlobStorageTestAccount.createTestAccount(AzureBlobStorageTestAccount.java:501)
> 	at org.apache.hadoop.fs.azure.AzureBlobStorageTestAccount.create(AzureBlobStorageTestAccount.java:522)
> 	at org.apache.hadoop.fs.azure.AzureBlobStorageTestAccount.create(AzureBlobStorageTestAccount.java:451)
> 	at org.apache.hadoop.fs.azure.TestNativeAzureFileSystemAuthorization.createTestAccount(TestNativeAzureFileSystemAuthorization.java:50)
> 	at org.apache.hadoop.fs.azure.AbstractWasbTestBase.setUp(AbstractWasbTestBase.java:47)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:497)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> 	at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:168)
> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> 	at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
> 	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> 	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
> 	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> 	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> {code}
> Actually, this error message is not very helpful. I'd admire you if you can immediately
find the root cause of this failure.
> Currently if the test *account* is missing, we simply skip the test with meaningful error
message. Here if the *account key* is missing, we should do the same thing: skip the test,
and tell the user why we skip it.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message