hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9607) Advance Hadoop Architecture (AHA) - HDFS Update (write-in-place)
Date Mon, 25 Jan 2016 10:32:39 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15115012#comment-15115012
] 

Steve Loughran commented on HDFS-9607:
--------------------------------------

no: the source should be a byte array with offset within the buffer and length in it.

Imagine you want to write 1GB from a 4GB source. creating a new byte array and copying over
1GB of data would be a performance killer. 

You need to consider where data come from, where it goes to. Often it's come over the network
as part of a larger byte[] payload. Sometimes it's then encrypted, usually into a pre-allocated
byte[] array. We don't need any more copies

> Advance Hadoop Architecture (AHA) - HDFS Update (write-in-place)
> ----------------------------------------------------------------
>
>                 Key: HDFS-9607
>                 URL: https://issues.apache.org/jira/browse/HDFS-9607
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Dinesh S. Atreya
>
> Link to Umbrella JIRA
> https://issues.apache.org/jira/browse/HADOOP-12620 
> Provide capability to carry out in-place writes/updates. Only writes in-place are supported
where the existing length does not change.
> For example, "Hello World" can be replaced by "Hello HDFS!"
> See https://issues.apache.org/jira/browse/HADOOP-12620?focusedCommentId=15046300&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15046300
for more details.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message