hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HADOOP-13968) S3a FS to support "__magic" path for the special "unmaterialized" writes
Date Wed, 06 Dec 2017 13:04:01 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-13968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Steve Loughran resolved HADOOP-13968.
       Resolution: Duplicate
         Assignee: Steve Loughran
    Fix Version/s: 3.1.0

> S3a FS to support "__magic" path for the special "unmaterialized" writes
> ------------------------------------------------------------------------
>                 Key: HADOOP-13968
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13968
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.0.0-beta1
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>             Fix For: 3.1.0
> S3AFileSystem to add support for a special path, such as  {{.temp_pending_put/}} or similar,
which, when used as the base of a path, indicates that the file is actually to be saved to
the parent dir, but only via a delayed put commit operation.
> At the same time, we may need blocks on some normal fileIO ops under these dirs, especially
rename and delete, as this would cause serious problems including data loss and large bills
for pending data.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org

View raw message