hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10880) Federation Mount Table State Store internal API
Date Fri, 28 Jul 2017 18:24:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16105436#comment-16105436
] 

Chris Douglas commented on HDFS-10880:
--------------------------------------

* {{MountTableResolver}} doesn't have a consistent locking model. Most operations use the
{{treeLock}} (e.g., when cleared) and {{cacheLock}} is only used in {{getDestinationForPath}}.
Similarly, results from the {{PathTree}} are resolved against the {{cache}} without holding
any locks, though both may be modified concurrently. Would a r/w lock be sufficient?
* In {{InvalidateLocationCache}}:
{noformat}
+    // TODO Determine next lexographic entry after source path
+    String nextSrc = path + "ZZZZ";
+    ConcurrentNavigableMap<String, PathLocation> subMap =
+        locationCache.subMap(path, nextSrc);
{noformat}
Instead of appending "ZZZZ" (which may not work for other locales), this can use a different
version of [subMap|https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ConcurrentNavigableMap.html#subMap-K-boolean-K-boolean-]
* Duplicate methods in {{PathLocation}}: {{getDestOrder}} and {{getDestinationOrder}}. Is
either necessary? This attribute is also persisted.
* {{PathLocation#prioritizeNamespace}} modifies the destination list under a lock, but the
list is accessed without locks within the class and through references that escape.
** This overrides the declared destination order for the record? It may simplify the design
to make each {{PathLocation}} immutable, and reorder them by replacing entries (unless they're
shared).
* This includes some classes (e.g., {{CachedRecordStore}}) also in HDFS-10687
* How are the comparators in {{MountTable}} used? Are these vestigial?
* The {{PathTree}} and {{PathTreeNode}} classes implement a quasi-Visitor pattern for maintaining
mount points. Are the traversal/query semantics used elsewhere? In this patch, it looks like
all the required operations could (exact-match, longest-match, etc.) could be implemented
less expensively in a [NavigableMap|https://docs.oracle.com/javase/8/docs/api/java/util/NavigableMap.html]

> Federation Mount Table State Store internal API
> -----------------------------------------------
>
>                 Key: HDFS-10880
>                 URL: https://issues.apache.org/jira/browse/HDFS-10880
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: fs
>            Reporter: Jason Kace
>            Assignee: Inigo Goiri
>         Attachments: HDFS-10880-HDFS-10467-000.patch, HDFS-10880-HDFS-10467-001.patch,
HDFS-10880-HDFS-10467-002.patch
>
>
> The Federation Mount Table State encapsulates the mapping of file paths in the global
namespace to a specific NN(nameservice) and local NN path.  The mount table is shared by all
router instances and represents a unified view of the global namespace.   The state store
API for the mount table allows the related records to be queried, updated and deleted.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message