hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ewan Higgs (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-13936) multipart upload to HDFS to support 0 byte upload
Date Mon, 24 Sep 2018 13:52:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-13936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16625847#comment-16625847
] 

Ewan Higgs commented on HDFS-13936:
-----------------------------------

Right, but you could touch a file with no implication of the file being open. This is used
in HDFS for sentinel files (not usually used for files you would want to concat).
{code:java}
hdfs dfs -touch /empty-file{code}
 

The best option seems to be to check if there is only a single part and if that single part
is empty, just touch the destination.

> multipart upload to HDFS to support 0 byte upload
> -------------------------------------------------
>
>                 Key: HDFS-13936
>                 URL: https://issues.apache.org/jira/browse/HDFS-13936
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: fs, hdfs
>    Affects Versions: 3.2.0
>            Reporter: Steve Loughran
>            Assignee: Ewan Higgs
>            Priority: Major
>
> MPUs to HDFS fail as you can't concat an empty block. 
> Whatever uploads to HDFS needs to recognise that specific case "0-byte file" and rather
than try and concat things, just create a 0-byte file there.
> Without this, you can't use MPU as a replacement for distcp or alternative commit protocols.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message