hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Demoor (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13568) S3AFastOutputStream to implement flush()
Date Thu, 08 Sep 2016 13:13:21 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15473830#comment-15473830

Thomas Demoor commented on HADOOP-13568:

This might be hard as only the last part of a MPU can be smaller then 5MB. So I fear that
the subsequent complete operation would fail if flush() is called when the buffer contains
less than 5MB.

http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadUploadPart.html ??Each part must be
at least 5 MB in size, except the last part. There is no size limit on the last part of your
multipart upload.??

> S3AFastOutputStream to implement flush()
> ----------------------------------------
>                 Key: HADOOP-13568
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13568
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Steve Loughran
>            Priority: Minor
> {{S3AFastOutputStream}} doesn't implement {{flush()}}, so it's a no-op.
> Really it should trigger a multipart upload of the current buffer.
> Note that simply calling {{uploadBuffer()} isn't enough...do that and things fail.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message