hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-13936) multipart upload to HDFS to support 0 byte upload
Date Wed, 26 Sep 2018 09:57:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-13936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16628495#comment-16628495
] 

Steve Loughran commented on HDFS-13936:
---------------------------------------

LGTM. 

The last round of specification & tests did have the empty upload test, so this patch
could be merged with that to combine the fix with the tests. It also tweaks the S3A MPU to
allow for 0-byte uploads too

> multipart upload to HDFS to support 0 byte upload
> -------------------------------------------------
>
>                 Key: HDFS-13936
>                 URL: https://issues.apache.org/jira/browse/HDFS-13936
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: fs, hdfs
>    Affects Versions: 3.2.0
>            Reporter: Steve Loughran
>            Assignee: Ewan Higgs
>            Priority: Major
>         Attachments: HDFS-13936.01.patch
>
>
> MPUs to HDFS fail as you can't concat an empty block. 
> Whatever uploads to HDFS needs to recognise that specific case "0-byte file" and rather
than try and concat things, just create a 0-byte file there.
> Without this, you can't use MPU as a replacement for distcp or alternative commit protocols.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message