Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Sat, 2 Aug 2014 00:09:38 +0000 (UTC)
From: "Colin Patrick McCabe (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12729973.1406327616071.100296.1406938178836@arcas>
In-Reply-To: <JIRA.12729973.1406327616071@arcas>
References: <JIRA.12729973.1406327616071@arcas>
Subject: [jira] [Commented] (HDFS-6757) Simplify lease manager with INodeID
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14083226#comment-14083226 ] 

Colin Patrick McCabe commented on HDFS-6757:
--------------------------------------------

bq. Perhaps I'm understanding. What is the use case for allowing a client to continue writing to a deleted file and have the results reflected in a snapshot? The lease is effectively for the mutable/non-snapshot path. If it's deleted, why wouldn't we want to revoke the lease? It's already "bad" enough that snapshotting a UC file does not record the visible length so the file in the snapshot continue to mutate, but continuing to modify a deleted file in a snapshot?

Sorry, I guess I phrased this poorly.  I'm not talking about writing to a snapshotted file, or anything like that.  I'm just commenting on the implementation detail that led to the need for the {{removedUCFiles}} list.  Basically, the issue is that an inode that is under construction may be part of a snapshot.  Just because we remove the snapshot doesn't mean we should revoke the lease on the under-construction file.  So we can't just do "if inode.isUnderconstruction \{ revoke_lease \}" inside the deletion functions.  We only want to revoke the lease when closing the file or removing it from the current reality.

bq. My mild concern/question is let's say this change is in 2.6. We deploy it, run a week, find a horrible problem and back up to 2.5 because it has the same layout.

2.5 can't replay edits from 2.6.  There's no forward-compatibility with edit logs, only backwards compatibility.  Basically the way things work in the rolling upgrade universe is that some changes can be downgraded cleanly, and others require "rollback" (i.e. revert the data back to the previous version).  HDFS-6757 is clearly a change that could be done via downgrade rather than rollback, since it's not adding any new data that 2.5 can't understand.

> Simplify lease manager with INodeID
> -----------------------------------
>
>                 Key: HDFS-6757
>                 URL: https://issues.apache.org/jira/browse/HDFS-6757
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Haohui Mai
>            Assignee: Haohui Mai
>         Attachments: HDFS-6757.000.patch, HDFS-6757.001.patch, HDFS-6757.002.patch, HDFS-6757.003.patch, HDFS-6757.004.patch
>
>
> Currently the lease manager records leases based on path instead of inode ids. Therefore, the lease manager needs to carefully keep track of the path of active leases during renames and deletes. This can be a non-trivial task.
> This jira proposes to simplify the logic by tracking leases using inodeids instead of paths.


--
This message was sent by Atlassian JIRA
(v6.2#6252)