hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Subroto Sanyal (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-5814) NativeS3FileSystem doesn't report progress when writing
Date Sat, 10 Sep 2011 05:27:08 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-5814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Subroto Sanyal updated HADOOP-5814:

    Attachment: HADOOP-5814.patch

The patch is on the trunk version.
The patch creates a input Stream which handles the Progressable reference and makes the call
periodically while reading from the underlying stream. The patch is tested in our cluster
and works fine.

> NativeS3FileSystem doesn't report progress when writing
> -------------------------------------------------------
>                 Key: HADOOP-5814
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5814
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs/s3
>            Reporter: Tom White
>            Assignee: Devaraj K
>         Attachments: HADOOP-5814.patch
> This results in timeouts since the whole file is uploaded in the close method. See http://www.mail-archive.com/core-user@hadoop.apache.org/msg09881.html.
> One solution is to keep a reference to the Progressable passed in to the NativeS3FsOutputStream's
constructor, and progress it during writes, and while copying the backup file to S3 in the
close method.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message