hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "wujinhu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15323) AliyunOSS: Improve copy file performance for AliyunOSSFileSystemStore
Date Thu, 03 Jan 2019 09:00:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732771#comment-16732771

wujinhu commented on HADOOP-15323:

Thanks [~cheersyang] I have added debug log and docs as you suggested.

We should add this feature to branch 2.x also, because some of our users still use hadoop
2.x now.

Please help to merge into those branches too. Many thanks.:)

> AliyunOSS: Improve copy file performance for AliyunOSSFileSystemStore
> ---------------------------------------------------------------------
>                 Key: HADOOP-15323
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15323
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/oss
>    Affects Versions: 2.10.0, 3.2.0, 3.1.1, 2.9.2, 3.0.3, 3.3.0
>            Reporter: wujinhu
>            Assignee: wujinhu
>            Priority: Major
>         Attachments: HADOOP-15323.001.patch, HADOOP-15323.002.patch, HADOOP-15323.003.patch
> Aliyun OSS will support shallow copy which means server will only copy metadata when
copy object operation occurs. 
> With shallow copy, we can use copyObject api instead of multi-part copy api if we do
not change object storage type & encryption type & source object  is uploaded by
Put / Multipart upload api.
> We will try to use copyObject api, and check result. If shallow copy disabled for this
object, then we will use multipart copy. So, I will remove fs.oss.multipart.upload.threshold

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message