hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9520) PeerCache evicts too frequently causing connection restablishments
Date Tue, 08 Dec 2015 18:02:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15047162#comment-15047162
] 

Chris Nauroth commented on HDFS-9520:
-------------------------------------

I don't think {{(capacity == multimap.keySet().size())}} would be correct.  If multiple threads
are calling the same DataNode concurrently, then there could be multiple sockets ({{Peer}}
instances) mapped to the same {{DatanodeID}} (basically the DN address) in the multi-map.
 {{dfs.client.socketcache.capacity}} limits the number of sockets in the cache (the values),
not the number of distinct DN addresses (the keys).

> PeerCache evicts too frequently causing connection restablishments
> ------------------------------------------------------------------
>
>                 Key: HDFS-9520
>                 URL: https://issues.apache.org/jira/browse/HDFS-9520
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Rajesh Balamohan
>         Attachments: HDFS-9520.png
>
>
> Env: 20 node setup
> dfs.client.socketcache.capacity = 16
> Issue:
> ======
> Monitored PeerCache and it was evicting lots of connections during close. Set "dfs.client.socketcache.capacity=20"
and tested again. Evictions still happened. Screenshot of profiler is attached in the JIRA.
> Workaround:
> ===========
> Temp fix was to set "dfs.client.socketcache.capacity=1000" to prevent eviction. 
> Added more debug logs revealed that multimap.size() was 40 instead of 20. LinkedListMultimap
returns the total values instead of key size causing lots of evictions.
> {code}
>    if (capacity == multimap.size()) {
>       evictOldest();
>     }
> {code}
> Should this be (capacity == multimap.keySet().size())  or is it expected that the "dfs.client.socketcache.capacity"
be set to very high value?
> \cc [~gopalv], [~sseth]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message