hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-13968) S3a FS to support "__magic" path for the special "unmaterialized" writes
Date Fri, 08 Sep 2017 17:48:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-13968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Steve Loughran updated HADOOP-13968:
    Summary: S3a FS to support "__magic" path for the special "unmaterialized" writes  (was:
S3a FS to support ".temp_pending_put" path for pending put operations)

> S3a FS to support "__magic" path for the special "unmaterialized" writes
> ------------------------------------------------------------------------
>                 Key: HADOOP-13968
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13968
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.0.0-beta1
>            Reporter: Steve Loughran
> S3AFileSystem to add support for a special path, such as  {{.temp_pending_put/}} or similar,
which, when used as the base of a path, indicates that the file is actually to be saved to
the parent dir, but only via a delayed put commit operation.
> At the same time, we may need blocks on some normal fileIO ops under these dirs, especially
rename and delete, as this would cause serious problems including data loss and large bills
for pending data.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message