hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10880) Federation Mount Table State Store internal API
Date Fri, 28 Jul 2017 18:24:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16105436#comment-16105436

Chris Douglas commented on HDFS-10880:

* {{MountTableResolver}} doesn't have a consistent locking model. Most operations use the
{{treeLock}} (e.g., when cleared) and {{cacheLock}} is only used in {{getDestinationForPath}}.
Similarly, results from the {{PathTree}} are resolved against the {{cache}} without holding
any locks, though both may be modified concurrently. Would a r/w lock be sufficient?
* In {{InvalidateLocationCache}}:
+    // TODO Determine next lexographic entry after source path
+    String nextSrc = path + "ZZZZ";
+    ConcurrentNavigableMap<String, PathLocation> subMap =
+        locationCache.subMap(path, nextSrc);
Instead of appending "ZZZZ" (which may not work for other locales), this can use a different
version of [subMap|https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ConcurrentNavigableMap.html#subMap-K-boolean-K-boolean-]
* Duplicate methods in {{PathLocation}}: {{getDestOrder}} and {{getDestinationOrder}}. Is
either necessary? This attribute is also persisted.
* {{PathLocation#prioritizeNamespace}} modifies the destination list under a lock, but the
list is accessed without locks within the class and through references that escape.
** This overrides the declared destination order for the record? It may simplify the design
to make each {{PathLocation}} immutable, and reorder them by replacing entries (unless they're
* This includes some classes (e.g., {{CachedRecordStore}}) also in HDFS-10687
* How are the comparators in {{MountTable}} used? Are these vestigial?
* The {{PathTree}} and {{PathTreeNode}} classes implement a quasi-Visitor pattern for maintaining
mount points. Are the traversal/query semantics used elsewhere? In this patch, it looks like
all the required operations could (exact-match, longest-match, etc.) could be implemented
less expensively in a [NavigableMap|https://docs.oracle.com/javase/8/docs/api/java/util/NavigableMap.html]

> Federation Mount Table State Store internal API
> -----------------------------------------------
>                 Key: HDFS-10880
>                 URL: https://issues.apache.org/jira/browse/HDFS-10880
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: fs
>            Reporter: Jason Kace
>            Assignee: Inigo Goiri
>         Attachments: HDFS-10880-HDFS-10467-000.patch, HDFS-10880-HDFS-10467-001.patch,
> The Federation Mount Table State encapsulates the mapping of file paths in the global
namespace to a specific NN(nameservice) and local NN path.  The mount table is shared by all
router instances and represents a unified view of the global namespace.   The state store
API for the mount table allows the related records to be queried, updated and deleted.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message