hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6757) Simplify lease manager with INodeID
Date Wed, 30 Jul 2014 23:13:39 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14080159#comment-14080159

Colin Patrick McCabe commented on HDFS-6757:

bq. [~jingzhao] wrote: With INode features we now no longer have INode replacement (exception
for INodeReference which is out of picture in this jira), thus I think it's fine to use INode
in the lease map. However, in the meanwhile, I'm not sure if this will increase the possibility
of memory leak due to failing to update the lease map (maybe because of buggy code).

I agree-- we might get some really nasty bugs by having a bunch of references to inodes that
somehow are "really" deleted here.  It would only require a minor mistake in the code.  The
memory savings doesn't seem worth it (and, of course, we're already reducing memory consumption
by switching away from strings)

bq. [~daryn] wrote: However, isn't a bug if there's no lease during a close op? Perhaps removeLeases
skips missing leases and calls a removeLease that requires a lease, and the close edit op
calls removeLease?

Imagine that a client gets SIGSTOP, and then its hard lease on its open-for-write file expires
and is removed.  The client can be resumed later, come back and then try to close.  So not
being able to find a lease during a close is certainly worthy of a nasty log message, but
I don't think we can call it a bug.

[~wheat9]:  I apologize if it is a dumb question, but do we need to pass removedUCFiles around
everywhere?  Can't we just look at the inodes in the lists we're passing and see if they're
under construction?  [~jingzhao]'s made a good point about how we don't want to remove leases
for open files just because we deleted an under-construction inode inside a snapshot.  But
we should be able to tell whether or not we're not removing the "current" version of the UC
inode somehow, right?

> Simplify lease manager with INodeID
> -----------------------------------
>                 Key: HDFS-6757
>                 URL: https://issues.apache.org/jira/browse/HDFS-6757
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Haohui Mai
>            Assignee: Haohui Mai
>         Attachments: HDFS-6757.000.patch, HDFS-6757.001.patch, HDFS-6757.002.patch, HDFS-6757.003.patch,
> Currently the lease manager records leases based on path instead of inode ids. Therefore,
the lease manager needs to carefully keep track of the path of active leases during renames
and deletes. This can be a non-trivial task.
> This jira proposes to simplify the logic by tracking leases using inodeids instead of

This message was sent by Atlassian JIRA

View raw message