hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14971) Merge S3A committers into trunk
Date Thu, 26 Oct 2017 19:07:01 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16221032#comment-16221032
] 

ASF GitHub Bot commented on HADOOP-14971:
-----------------------------------------

Github user steveloughran commented on a diff in the pull request:

    https://github.com/apache/hadoop/pull/282#discussion_r147238117
  
    --- Diff: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
---
    @@ -1130,17 +1359,23 @@ private void blockRootDelete(String key) throws InvalidRequestException
{
        * Perform a bulk object delete operation.
        * Increments the {@code OBJECT_DELETE_REQUESTS} and write
        * operation statistics.
    +   * Retry policy: retry untranslated; delete considered idempotent.
        * @param deleteRequest keys to delete on the s3-backend
        * @throws MultiObjectDeleteException one or more of the keys could not
        * be deleted.
        * @throws AmazonClientException amazon-layer failure.
        */
    +  @Retries.RetryRaw
       private void deleteObjects(DeleteObjectsRequest deleteRequest)
    -      throws MultiObjectDeleteException, AmazonClientException {
    +      throws MultiObjectDeleteException, AmazonClientException, IOException {
         incrementWriteOperations();
    -    incrementStatistic(OBJECT_DELETE_REQUESTS, 1);
         try {
    -      s3.deleteObjects(deleteRequest);
    +      invoker.retryUntranslated("delete",
    --- End diff --
    
    OK. How about: no retry/untranslated in innerDelete, -> retryTranslated in API delete,
with anything the committer doing wrapping it in its own retries and/or ignoring of failures
(which of course, is  a valid option in some cases)


> Merge S3A committers into trunk
> -------------------------------
>
>                 Key: HADOOP-14971
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14971
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.0.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>
> Merge the HADOOP-13786 committer into trunk. This branch is being set up as a github
PR for review there & to keep it out the mailboxes of the watchers on the main JIRA



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message