hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14971) Merge S3A committers into trunk
Date Fri, 27 Oct 2017 12:14:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16222255#comment-16222255
] 

ASF GitHub Bot commented on HADOOP-14971:
-----------------------------------------

Github user steveloughran commented on a diff in the pull request:

    https://github.com/apache/hadoop/pull/282#discussion_r147394432
  
    --- Diff: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
---
    @@ -1028,60 +1183,98 @@ public S3AStorageStatistics getStorageStatistics() {
     
       /**
        * Request object metadata; increments counters in the process.
    +   * Retry policy: retry untranslated.
        * @param key key
        * @return the metadata
    +   * @throws IOException if the retry invocation raises one (it shouldn't).
        */
    -  protected ObjectMetadata getObjectMetadata(String key) {
    -    incrementStatistic(OBJECT_METADATA_REQUESTS);
    +  @Retries.RetryRaw
    +  protected ObjectMetadata getObjectMetadata(String key) throws IOException {
         GetObjectMetadataRequest request =
             new GetObjectMetadataRequest(bucket, key);
         //SSE-C requires to be filled in if enabled for object metadata
         if(S3AEncryptionMethods.SSE_C.equals(serverSideEncryptionAlgorithm) &&
             StringUtils.isNotBlank(getServerSideEncryptionKey(getConf()))){
           request.setSSECustomerKey(generateSSECustomerKey());
         }
    -    ObjectMetadata meta = s3.getObjectMetadata(request);
    +    ObjectMetadata meta = invoker.retryUntranslated(
    +        "GET " + key,
    +        true,
    +        () -> {
    +          incrementStatistic(OBJECT_METADATA_REQUESTS);
    +          return s3.getObjectMetadata(request);
    +        });
         incrementReadOperations();
         return meta;
       }
     
       /**
        * Initiate a {@code listObjects} operation, incrementing metrics
        * in the process.
    +   *
    +   * Retry policy: retry untranslated.
        * @param request request to initiate
        * @return the results
    +   * @throws IOException if the retry invocation raises one (it shouldn't).
        */
    -  protected S3ListResult listObjects(S3ListRequest request) {
    -    incrementStatistic(OBJECT_LIST_REQUESTS);
    +  @Retries.RetryRaw
    +  protected S3ListResult listObjects(S3ListRequest request) throws IOException {
         incrementReadOperations();
    +    incrementStatistic(OBJECT_LIST_REQUESTS);
    +    validateListArguments(request);
    +    return invoker.retryUntranslated(
    +        request.toString(),
    +        true,
    +        () -> {
    +          if (useListV1) {
    +            Preconditions.checkArgument(request.isV1());
    --- End diff --
    
    done


> Merge S3A committers into trunk
> -------------------------------
>
>                 Key: HADOOP-14971
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14971
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.0.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>
> Merge the HADOOP-13786 committer into trunk. This branch is being set up as a github
PR for review there & to keep it out the mailboxes of the watchers on the main JIRA



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message