hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7056) Snapshot support for truncate
Date Fri, 14 Nov 2014 07:51:34 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14211988#comment-14211988
] 

Konstantin Shvachko commented on HDFS-7056:
-------------------------------------------

Colin thank you for the review.

??I don't have MS Windows or MS Office. Maybe you can attach a PDF at some point???

Guo is the owner of the document.
I use LibreOffice, which shows me the .docx file including all pictures. No Windows.
On my side, I updated the design doc for truncate to reflect snapshots and attached the pdf
file under HDFS-3107.

??Did you consider a design where the inode ID of the truncated file changes???

I see what you mean. But that would result in creating a new inode for each truncate op under
a snapshot. You are right it will use more RAM.

??I'd prefer not to add this method, since it's only used once in a unit test. And we already
have Block#equals and Block#compareTo.??

sameAs() is indeed used only once, but in FSDirectory, not a unit test. It could have been
used in a few more places, where we check that two blocks have the same ids, genStamps and
lengths (that is are the same). We do have equals(), compareTo(), and matchingIdAndGenStamp().
But they are different from sameAs() as the first two compare only blockIds, and the third
adds genStamps.
Not that it is imperative for the truncate feature, just explaining how it could have been
useful.

??If the client calls recoverLease on this file that is being truncated??

Agree with Plamen. This is no different from create or append. That is, if the lease has not
expired yet, but recoverLease() is called
* by a different client, then the operation will fail (with a rather awkward exception).
* by the same client, then the truncate recovery will be re-triggered.

If the lease expired by the time recoverLease() is called, then
* it is a no op, when file is already closed,
* and it will re-trigger the recovery when the file is still under construction.

> Snapshot support for truncate
> -----------------------------
>
>                 Key: HDFS-7056
>                 URL: https://issues.apache.org/jira/browse/HDFS-7056
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>    Affects Versions: 3.0.0
>            Reporter: Konstantin Shvachko
>            Assignee: Plamen Jeliazkov
>         Attachments: HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch,
HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch,
HDFS-7056.patch, HDFS-7056.patch, HDFS-7056.patch, HDFS-7056.patch, HDFS-7056.patch, HDFS-7056.patch,
HDFSSnapshotWithTruncateDesign.docx
>
>
> Implementation of truncate in HDFS-3107 does not allow truncating files which are in
a snapshot. It is desirable to be able to truncate and still keep the old file state of the
file in the snapshot.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message