hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Plamen Jeliazkov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3107) HDFS truncate
Date Sat, 27 Sep 2014 17:57:34 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14150681#comment-14150681
] 

Plamen Jeliazkov commented on HDFS-3107:
----------------------------------------

The FindBugs addition seems to be for a class that is unmodified by my patch:
"Class org.apache.hadoop.hdfs.protocol.datatransfer.ReplaceDatanodeOnFailure$Policy defines
non-transient non-serializable instance field condition"

I am not sure how that happened. I could remedy it by adding a transient modifier to the Condition
field 'condition', but it seems that it is unrelated to the truncate patch. 
Probably best to keep the separation of concerns. 

The failure in TestEncryptionZonesWithKMS was due to a Mini HDFS Cluster setup timing out.
I have ran TestEncryptionZonesWithKMS locally and seen it succeed with my patch applied.

If you are going to commit be aware that you have to also commit the attached 'editsStored'
file into the hadoop-hdfs-project/hadoop-hdfs/src/test/resources directory and overwrite the
file that is already there.

> HDFS truncate
> -------------
>
>                 Key: HDFS-3107
>                 URL: https://issues.apache.org/jira/browse/HDFS-3107
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: datanode, namenode
>            Reporter: Lei Chang
>            Assignee: Plamen Jeliazkov
>         Attachments: HDFS-3107.patch, HDFS-3107.patch, HDFS-3107.patch, HDFS-3107.patch,
HDFS-3107.patch, HDFS_truncate.pdf, HDFS_truncate_semantics_Mar15.pdf, HDFS_truncate_semantics_Mar21.pdf,
editsStored
>
>   Original Estimate: 1,344h
>  Remaining Estimate: 1,344h
>
> Systems with transaction support often need to undo changes made to the underlying storage
when a transaction is aborted. Currently HDFS does not support truncate (a standard Posix
operation) which is a reverse operation of append, which makes upper layer applications use
ugly workarounds (such as keeping track of the discarded byte range per file in a separate
metadata store, and periodically running a vacuum process to rewrite compacted files) to overcome
this limitation of HDFS.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message