hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dan Hecht (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-11570) S3AInputStream.close() downloads the remaining bytes of the object from S3
Date Tue, 10 Feb 2015 06:55:35 GMT
Dan Hecht created HADOOP-11570:
----------------------------------

             Summary: S3AInputStream.close() downloads the remaining bytes of the object from
S3
                 Key: HADOOP-11570
                 URL: https://issues.apache.org/jira/browse/HADOOP-11570
             Project: Hadoop Common
          Issue Type: Bug
          Components: fs/s3
    Affects Versions: 2.6.0
            Reporter: Dan Hecht


Currently, S3AInputStream.close() calls S3Object.close().  But, S3Object.close() will read
the remaining bytes of the S3 object, potentially transferring a lot of bytes from S3 that
are discarded.  Instead, the wrapped stream should be aborted to avoid transferring discarded
bytes (unless the preceding read() finished at contentLength).  For example, reading only
the first byte of a 1 GB object and then closing the stream will result in all 1 GB transferred
from S3.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message