hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-14547) [WASB] the configured retry policy is not used for all storage operations.
Date Mon, 19 Jun 2017 23:17:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-14547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Thomas updated HADOOP-14547:
----------------------------
    Attachment: HADOOP-14547.001.patch

Attaching patch HADOOP-14547.001.patch.

The issue is fixed by reading the WASB configuration settings before creating and initializing
the storage components (CloudBlobClient, CloudBlobContainer, etc).  The retry policy and session
timeout are now stored by the StorageInterface implementations and used to properly initialize
the storage objects.

Unfortunately, the retry policy cannot easily be validated with unit tests due to functionality
that is not exposed by the Azure Storage SDK.  I have carefully validated this change in the
debugger to ensure that the user configured retry policy is used.

> [WASB] the configured retry policy is not used for all storage operations.
> --------------------------------------------------------------------------
>
>                 Key: HADOOP-14547
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14547
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/azure
>    Affects Versions: 2.9.0, 3.0.0-alpha4
>            Reporter: Thomas
>            Assignee: Thomas
>             Fix For: 2.9.0, 3.0.0-alpha4
>
>         Attachments: HADOOP-14547.001.patch
>
>
> There are a few places where the WASB retry policy was not used, and instead the Azure
Storage SDK default retry policy was used.  These places include some calls to blob.exist(),
container.exists(), blob.delete(), and operations with secure mode enabled (fs.azure.secure.mode
= true).  
> You can set a break point on com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry
in the Azure Storage SDK and see that the WASB configured retry policy (DEFAULT_MAX_RETRY_ATTEMPTS,
etc.) is not used.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message