hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-7056) Snapshot support for truncate
Date Tue, 23 Dec 2014 12:24:15 GMT

     [ https://issues.apache.org/jira/browse/HDFS-7056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Konstantin Shvachko updated HDFS-7056:
--------------------------------------
    Attachment: HDFS-7056-15.patch

# When the last snapshot is deleted it doesn't even go into {{combineAndCollectSnapshotBlocks()}}.
It cleans everything via directory's deleted list. See {{destroyDeletedList()}}, which is
called from {{destroyDiffAndCollectBlocks()}}. The patch did not change anything here.
So passing null into {{collectBlocksAndClear()}} is not a problem. But I agree it is more
logical, so I added the parameter. I also added a test case to make sure that the number of
inodes is decreasing in this case.
# ??findEarlierSnapshotBlocks is not easy to follow.??
You seem to be doing alright. :-) And snapshots are not easy to follow in the first place.
I think I simplified and optimized it.
# ??findLaterSnapshotBlocks is always coupled with an extra null check??
I considered it. It will require passing INodeFile as a parameter into{{findLaterSnapshotBlocks()}},
which to me unnecessarily breaks the hierarchy of classes. We would be calling a method on
the INodeFile member, which in turn will call a method of INodeFile. I would rather avoid
such cyclicals if possible, which I know we use everywhere.
# ??Could you generally explain why we need to bump up the NN layout version here???
## preventing rolling downgrade is one thing
## If you do regular startup without rolling upgrade, then you should be prompted to use upgrade
option. Otherwise, you loose opportunity to rollback if needed.
## I thought that knowing which version supports truncate will be useful for rolling upgrades
from pre-truncate versions. Say if you upgrade active NN before SBN and call truncate, SBN
will crash. So it may be better not to use new functionality until the upgrade is done.

Was looking at the Jenkins warning from the last build.
- Turned out one findbugs warning is related to truncate. Fixed it in HDFS-3107.
- The other warning is covered in HDFS-7551.
- TestSymlinkHdfsFileContext failure is related to HADOOP-11432
- TestOEV is expected.
- Could not reproduce TestFileTruncate wait timeout. Will keep an eye if it happens again.

> Snapshot support for truncate
> -----------------------------
>
>                 Key: HDFS-7056
>                 URL: https://issues.apache.org/jira/browse/HDFS-7056
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>    Affects Versions: 3.0.0
>            Reporter: Konstantin Shvachko
>            Assignee: Plamen Jeliazkov
>         Attachments: HDFS-3107-HDFS-7056-combined-13.patch, HDFS-3107-HDFS-7056-combined.patch,
HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch,
HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch,
HDFS-3107-HDFS-7056-combined.patch, HDFS-3107-HDFS-7056-combined.patch, HDFS-7056-13.patch,
HDFS-7056-15.patch, HDFS-7056.patch, HDFS-7056.patch, HDFS-7056.patch, HDFS-7056.patch, HDFS-7056.patch,
HDFS-7056.patch, HDFS-7056.patch, HDFS-7056.patch, HDFSSnapshotWithTruncateDesign.docx
>
>
> Implementation of truncate in HDFS-3107 does not allow truncating files which are in
a snapshot. It is desirable to be able to truncate and still keep the old file state of the
file in the snapshot.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message