hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gian Merlino (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-10737) S3n silent failure on copy, data loss on rename
Date Mon, 23 Jun 2014 17:19:25 GMT
Gian Merlino created HADOOP-10737:

             Summary: S3n silent failure on copy, data loss on rename
                 Key: HADOOP-10737
                 URL: https://issues.apache.org/jira/browse/HADOOP-10737
             Project: Hadoop Common
          Issue Type: Bug
          Components: fs/s3
    Affects Versions: 2.4.0
            Reporter: Gian Merlino

Jets3tNativeFileSystemStore.copy(String, String) handles its exceptions with handleServiceException(String,
ServiceException), which behaves like:

1) Throw FileNotFoundException if the exception's error code is NoSuchKey
2) Otherwise, throw IOException if the exception's cause is an IOException
3) Otherwise, LOG.debug a message and throw nothing

So S3 exceptions other than NoSuchKey (like RequestTimeout, ServiceUnavailable) are suppressed.
This makes "copy" fail while still returning as if it succeeded. Furthermore since NativeS3FileSystem's
"rename" is implemented as a copy followed by a delete, this means "rename" can delete the
source key even though the copy has failed.

This message was sent by Atlassian JIRA

View raw message