hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3107) HDFS truncate
Date Fri, 31 Oct 2014 01:05:37 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191142#comment-14191142
] 

Konstantin Shvachko commented on HDFS-3107:
-------------------------------------------

Let me try to clarify.
# The latest attached patch (Oct 17) does not support snapshots. We are targeting to post
a patch for truncate with snapshots under HDFS-7056. We are in the testing stage there.
# I did comment on rollbacks [just yesterday|https://issues.apache.org/jira/browse/HDFS-3107?focusedCommentId=14189216&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14189216].
Copy replica on truncate solves the issue. Also addressed in the comming patch under HDFS-7056.
# The solution [you described in your comment|https://issues.apache.org/jira/browse/HDFS-3107?focusedCommentId=14190706&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14190706]
is exactly the one we are implementing here. Or I do not understand how it differs from the
design document.

??Can we put numbers on patch files? I find it difficult to keep track of them when they all
have the same file name.??
~Colin I usually sort attachments by date. That way you do need to track any file names or
numbers. Actually was asking many times to change the default for sorting the attachements,
but never got a solution for that. Sould have been so much easier to just look at them in
the order of submission, same as comments.~

> HDFS truncate
> -------------
>
>                 Key: HDFS-3107
>                 URL: https://issues.apache.org/jira/browse/HDFS-3107
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: datanode, namenode
>            Reporter: Lei Chang
>            Assignee: Plamen Jeliazkov
>         Attachments: HDFS-3107.008.patch, HDFS-3107.patch, HDFS-3107.patch, HDFS-3107.patch,
HDFS-3107.patch, HDFS-3107.patch, HDFS-3107.patch, HDFS-3107.patch, HDFS_truncate.pdf, HDFS_truncate.pdf,
HDFS_truncate_semantics_Mar15.pdf, HDFS_truncate_semantics_Mar21.pdf, editsStored, editsStored,
editsStored.xml
>
>   Original Estimate: 1,344h
>  Remaining Estimate: 1,344h
>
> Systems with transaction support often need to undo changes made to the underlying storage
when a transaction is aborted. Currently HDFS does not support truncate (a standard Posix
operation) which is a reverse operation of append, which makes upper layer applications use
ugly workarounds (such as keeping track of the discarded byte range per file in a separate
metadata store, and periodically running a vacuum process to rewrite compacted files) to overcome
this limitation of HDFS.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message