hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lei (Eddy) Xu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-7705) {{FileSystem}} should expose some performance characteristics for caller (e.g., FsShell) to choose right algorithm.
Date Thu, 29 Jan 2015 22:05:36 GMT
Lei (Eddy) Xu created HDFS-7705:
-----------------------------------

             Summary: {{FileSystem}} should expose some performance characteristics for caller
(e.g., FsShell) to choose right algorithm.
                 Key: HDFS-7705
                 URL: https://issues.apache.org/jira/browse/HDFS-7705
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: tools
    Affects Versions: 2.6.0
            Reporter: Lei (Eddy) Xu


When running {{hadoop fs -put}},  {{FsShell}} creates a {{._COPYING_.}} file on the target
directory, and then renames it to target file when the write is done. However, for some targeted
systems, such as S3, Azure and Swift, a partial failure write request (i.e., {{PUT}}) has
not side effect, while the {{rename}} operation is expensive. 

{{FileSystem}} should expose some characteristics so that the operation such as {{CommandWithDestination#copyStreamToTarget()}}
can detect and choose the right way to do.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message