hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Musaddique Hossain (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-13991) Retry management in NativeS3FileSystem to avoid file upload problem
Date Sat, 14 Jan 2017 18:44:26 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-13991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Musaddique Hossain updated HADOOP-13991:
----------------------------------------
    Description: 
NativeS3FileSystem does not support any retry management for failed uploading to S3.

If due to socket timeout or any other network exception, file uploading to S3 bucket fails,
then uploading fails and temporary file gets deleted.

This can be solved by using asynchronous retry management.

We have made following modifications to NativeS3FileSystem to add the retry management, which
is working fine in our product system, without any uploading failure:

{code:title=NativeS3FileSystem.java|borderStyle=solid}
@@ -36,6 +36,7 @@
 import java.util.Map;
 import java.util.Set;
 import java.util.TreeSet;
+import java.util.concurrent.Callable;
 import java.util.concurrent.TimeUnit;

 import com.google.common.base.Preconditions;
@@ -279,9 +280,19 @@
       backupStream.close();
       LOG.info("OutputStream for key '{}' closed. Now beginning upload", key);

+      Callable<Void> task = new Callable<Void>() {
+         private final byte[] md5Hash = digest == null ? null : digest.digest();
+         public Void call() throws IOException {
+            store.storeFile(key, backupFile, md5Hash);
+            return null;
+         }
+      };
+      RetriableTask<Void> r = new RetriableTask<Void>(task);
+      
       try {
-        byte[] md5Hash = digest == null ? null : digest.digest();
-        store.storeFile(key, backupFile, md5Hash);
+         r.call();
+      } catch (Exception e) {
+         throw new IOException(e);
       } finally {
         if (!backupFile.delete()) {
           LOG.warn("Could not delete temporary s3n file: " + backupFile);

{code}

  was:
NativeS3FileSystem does not support any retry management for failed uploading to S3.

If due to socket timeout or any other network exception, file uploading to S3 bucket fails,
then uploading fails and temporary file gets deleted.

This can be solved by using asynchronous retry management.

We have made following modifications to NativeS3FileSystem to add the retry management, which
is working fine in our product system, without any uploading failure:

@@ -36,6 +36,7 @@
 import java.util.Map;
 import java.util.Set;
 import java.util.TreeSet;
+import java.util.concurrent.Callable;
 import java.util.concurrent.TimeUnit;

 import com.google.common.base.Preconditions;
@@ -279,9 +280,19 @@
       backupStream.close();
       LOG.info("OutputStream for key '{}' closed. Now beginning upload", key);

+      Callable<Void> task = new Callable<Void>() {
+         private final byte[] md5Hash = digest == null ? null : digest.digest();
+         public Void call() throws IOException {
+            store.storeFile(key, backupFile, md5Hash);
+            return null;
+         }
+      };
+      RetriableTask<Void> r = new RetriableTask<Void>(task);
+      
       try {
-        byte[] md5Hash = digest == null ? null : digest.digest();
-        store.storeFile(key, backupFile, md5Hash);
+         r.call();
+      } catch (Exception e) {
+         throw new IOException(e);
       } finally {
         if (!backupFile.delete()) {
           LOG.warn("Could not delete temporary s3n file: " + backupFile);



> Retry management in NativeS3FileSystem to avoid file upload problem
> -------------------------------------------------------------------
>
>                 Key: HADOOP-13991
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13991
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs/s3
>    Affects Versions: 2.7.3
>            Reporter: Musaddique Hossain
>            Priority: Minor
>
> NativeS3FileSystem does not support any retry management for failed uploading to S3.
> If due to socket timeout or any other network exception, file uploading to S3 bucket
fails, then uploading fails and temporary file gets deleted.
> This can be solved by using asynchronous retry management.
> We have made following modifications to NativeS3FileSystem to add the retry management,
which is working fine in our product system, without any uploading failure:
> {code:title=NativeS3FileSystem.java|borderStyle=solid}
> @@ -36,6 +36,7 @@
>  import java.util.Map;
>  import java.util.Set;
>  import java.util.TreeSet;
> +import java.util.concurrent.Callable;
>  import java.util.concurrent.TimeUnit;
>  import com.google.common.base.Preconditions;
> @@ -279,9 +280,19 @@
>        backupStream.close();
>        LOG.info("OutputStream for key '{}' closed. Now beginning upload", key);
> +      Callable<Void> task = new Callable<Void>() {
> +         private final byte[] md5Hash = digest == null ? null : digest.digest();
> +         public Void call() throws IOException {
> +            store.storeFile(key, backupFile, md5Hash);
> +            return null;
> +         }
> +      };
> +      RetriableTask<Void> r = new RetriableTask<Void>(task);
> +      
>        try {
> -        byte[] md5Hash = digest == null ? null : digest.digest();
> -        store.storeFile(key, backupFile, md5Hash);
> +         r.call();
> +      } catch (Exception e) {
> +         throw new IOException(e);
>        } finally {
>          if (!backupFile.delete()) {
>            LOG.warn("Could not delete temporary s3n file: " + backupFile);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message