hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lei (Eddy) Xu (JIRA)" <j...@apache.org>
Subject [jira] [Moved] (HADOOP-11525) FileSystem should expose some performance characteristics for caller (e.g., FsShell) to choose the right algorithm.
Date Fri, 30 Jan 2015 00:40:34 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-11525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Lei (Eddy) Xu moved HDFS-7705 to HADOOP-11525:
----------------------------------------------

          Component/s:     (was: tools)
                       tools
     Target Version/s: 2.7.0  (was: 3.0.0, 2.7.0)
    Affects Version/s:     (was: 2.6.0)
                       2.6.0
                  Key: HADOOP-11525  (was: HDFS-7705)
              Project: Hadoop Common  (was: Hadoop HDFS)

> FileSystem should expose some performance characteristics for caller (e.g., FsShell)
to choose the right algorithm.
> -------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-11525
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11525
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: tools
>    Affects Versions: 2.6.0
>            Reporter: Lei (Eddy) Xu
>
> When running {{hadoop fs -put}},  {{FsShell}} creates a {{._COPYING_.}} file on the target
directory, and then renames it to target file when the write is done. However, for some targeted
systems, such as S3, Azure and Swift, a partial failure write request (i.e., {{PUT}}) has
not side effect, while the {{rename}} operation is expensive. 
> {{FileSystem}} should expose some characteristics so that the operation such as {{CommandWithDestination#copyStreamToTarget()}}
can detect and choose the right way to do.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message