hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Virajith Jalaparti (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12802) RBF: Control MountTableResolver cache size
Date Thu, 04 Jan 2018 19:12:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16311876#comment-16311876

Virajith Jalaparti commented on HDFS-12802:

[~elgoiri] thanks for posting this. 

A couple of comments:
# In {{MountTableResolver#startCacheCleaner}}, there are two calls to {{Executors.newScheduledThreadPool}}.
I think only one is needed.
# The added test case ({{testCacheCleaning}}) will not pass as the configuration parameters
({{MAX_CACHE_TIME_KEY}} and {{MAX_CACHE_SIZE_KEY}}) aren't set correctly for this particular
test. {{MAX_CACHE_SIZE_KEY}} should be set to 10 for this test case to pass. Also, in the
call to {{GenericTestUtils.waitFor}}, can we use the {{MAX_CACHE_TIME_KEY}} as the wait time?
I would say use 2* {{MAX_CACHE_TIME_KEY}} to make sure it will pass.

Minor: can we call {{MAX_CACHE_TIME_KEY}} as {{MAX_CACHE_TIME_MILLIS_KEY}} so that it is clearer
the time is in millis? Also, the first call to {{mountTable.getCacheSize()}} in {{testCacheCleaning}}
is not required.

> RBF: Control MountTableResolver cache size
> ------------------------------------------
>                 Key: HDFS-12802
>                 URL: https://issues.apache.org/jira/browse/HDFS-12802
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Íñigo Goiri
>            Assignee: Íñigo Goiri
>         Attachments: HDFS-12802.000.patch, HDFS-12802.001.patch
> Currently, the {{MountTableResolver}} caches the resolutions for the {{PathLocation}}.
However, this cache can grow with no limits if there are a lot of unique paths. Some of these
cached resolutions might not be used at all.
> The {{MountTableResolver}} should clean the {{locationCache}} periodically.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message