hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5203) Concurrent clients that add a cache directive on the same path may prematurely uncache from each other.
Date Fri, 18 Oct 2013 19:12:42 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13799422#comment-13799422
] 

Colin Patrick McCabe commented on HDFS-5203:
--------------------------------------------

Thanks for tackling this, Chris.

bq. Remove CacheManager enforcement of uniqueness of descriptors for path + pool.

great

bq. Add API for DistributedFileSystem#removePathBasedCacheDescriptors given a Path. Implement
this all the way down through DFSClient, protobuf, FSNamesystem, and CacheManager.

I don't think we need this.  Why not just have the shell do a listPathBasedCacheDescriptors,
and then send relevant removes  listPathBasedCacheDescriptors takes an optional path argument
so that the client doesn't have to be bothered with seeing results that don't pertain to the
given path.

We got rid of the batch operations because they were a big headache in general.  i.e. what
happens if one remove fails, but the others don't?  How can the client get information about
each specific remove that happened?  What if the client has permission to remove one but not
the others?

> Concurrent clients that add a cache directive on the same path may prematurely uncache
from each other.
> -------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-5203
>                 URL: https://issues.apache.org/jira/browse/HDFS-5203
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: HDFS-4949
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
>         Attachments: HDFS-5203.1.patch
>
>
> When a client adds a cache directive, we assign it a unique ID and return that ID to
the client.  If multiple clients add a cache directive for the same path, then we return the
same ID.  If one client then removes the cache entry for that ID, then it is removed for all
clients.  Then, when this change becomes visible in subsequent cache reports, the datanodes
may {{munlock}} the block before the other clients are done with it.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message