hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6757) Simplify lease manager with INodeID
Date Sat, 02 Aug 2014 00:09:38 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14083226#comment-14083226

Colin Patrick McCabe commented on HDFS-6757:

bq. Perhaps I'm understanding. What is the use case for allowing a client to continue writing
to a deleted file and have the results reflected in a snapshot? The lease is effectively for
the mutable/non-snapshot path. If it's deleted, why wouldn't we want to revoke the lease?
It's already "bad" enough that snapshotting a UC file does not record the visible length so
the file in the snapshot continue to mutate, but continuing to modify a deleted file in a

Sorry, I guess I phrased this poorly.  I'm not talking about writing to a snapshotted file,
or anything like that.  I'm just commenting on the implementation detail that led to the need
for the {{removedUCFiles}} list.  Basically, the issue is that an inode that is under construction
may be part of a snapshot.  Just because we remove the snapshot doesn't mean we should revoke
the lease on the under-construction file.  So we can't just do "if inode.isUnderconstruction
\{ revoke_lease \}" inside the deletion functions.  We only want to revoke the lease when
closing the file or removing it from the current reality.

bq. My mild concern/question is let's say this change is in 2.6. We deploy it, run a week,
find a horrible problem and back up to 2.5 because it has the same layout.

2.5 can't replay edits from 2.6.  There's no forward-compatibility with edit logs, only backwards
compatibility.  Basically the way things work in the rolling upgrade universe is that some
changes can be downgraded cleanly, and others require "rollback" (i.e. revert the data back
to the previous version).  HDFS-6757 is clearly a change that could be done via downgrade
rather than rollback, since it's not adding any new data that 2.5 can't understand.

> Simplify lease manager with INodeID
> -----------------------------------
>                 Key: HDFS-6757
>                 URL: https://issues.apache.org/jira/browse/HDFS-6757
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Haohui Mai
>            Assignee: Haohui Mai
>         Attachments: HDFS-6757.000.patch, HDFS-6757.001.patch, HDFS-6757.002.patch, HDFS-6757.003.patch,
> Currently the lease manager records leases based on path instead of inode ids. Therefore,
the lease manager needs to carefully keep track of the path of active leases during renames
and deletes. This can be a non-trivial task.
> This jira proposes to simplify the logic by tracking leases using inodeids instead of

This message was sent by Atlassian JIRA

View raw message