Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Fri, 28 Jul 2017 18:24:00 +0000 (UTC)
From: "Chris Douglas (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.13006333.1474413544000.39795.1501266240089@Atlassian.JIRA>
In-Reply-To: <JIRA.13006333.1474413544000@Atlassian.JIRA>
References: <JIRA.13006333.1474413544000@Atlassian.JIRA> <JIRA.13006333.1474413544367@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HDFS-10880) Federation Mount Table State Store
 internal API
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 28 Jul 2017 18:24:05 -0000


    [ https://issues.apache.org/jira/browse/HDFS-10880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16105436#comment-16105436 ] 

Chris Douglas commented on HDFS-10880:
--------------------------------------

* {{MountTableResolver}} doesn't have a consistent locking model. Most operations use the {{treeLock}} (e.g., when cleared) and {{cacheLock}} is only used in {{getDestinationForPath}}. Similarly, results from the {{PathTree}} are resolved against the {{cache}} without holding any locks, though both may be modified concurrently. Would a r/w lock be sufficient?
* In {{InvalidateLocationCache}}:
{noformat}
+    // TODO Determine next lexographic entry after source path
+    String nextSrc = path + "ZZZZ";
+    ConcurrentNavigableMap<String, PathLocation> subMap =
+        locationCache.subMap(path, nextSrc);
{noformat}
Instead of appending "ZZZZ" (which may not work for other locales), this can use a different version of [subMap|https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ConcurrentNavigableMap.html#subMap-K-boolean-K-boolean-]
* Duplicate methods in {{PathLocation}}: {{getDestOrder}} and {{getDestinationOrder}}. Is either necessary? This attribute is also persisted.
* {{PathLocation#prioritizeNamespace}} modifies the destination list under a lock, but the list is accessed without locks within the class and through references that escape.
** This overrides the declared destination order for the record? It may simplify the design to make each {{PathLocation}} immutable, and reorder them by replacing entries (unless they're shared).
* This includes some classes (e.g., {{CachedRecordStore}}) also in HDFS-10687
* How are the comparators in {{MountTable}} used? Are these vestigial?
* The {{PathTree}} and {{PathTreeNode}} classes implement a quasi-Visitor pattern for maintaining mount points. Are the traversal/query semantics used elsewhere? In this patch, it looks like all the required operations could (exact-match, longest-match, etc.) could be implemented less expensively in a [NavigableMap|https://docs.oracle.com/javase/8/docs/api/java/util/NavigableMap.html]

> Federation Mount Table State Store internal API
> -----------------------------------------------
>
>                 Key: HDFS-10880
>                 URL: https://issues.apache.org/jira/browse/HDFS-10880
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: fs
>            Reporter: Jason Kace
>            Assignee: Inigo Goiri
>         Attachments: HDFS-10880-HDFS-10467-000.patch, HDFS-10880-HDFS-10467-001.patch, HDFS-10880-HDFS-10467-002.patch
>
>
> The Federation Mount Table State encapsulates the mapping of file paths in the global namespace to a specific NN(nameservice) and local NN path.  The mount table is shared by all router instances and represents a unified view of the global namespace.   The state store API for the mount table allows the related records to be queried, updated and deleted.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org