hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12475) Replace guava Cache with ConcurrentHashMap for caching Connection in ipc Client
Date Wed, 14 Oct 2015 15:42:06 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14957106#comment-14957106
] 

Sangjin Lee commented on HADOOP-12475:
--------------------------------------

+1 on [~Apache9]'s comment. A more correct way of using the ConcurrentHashMap is to utilize
its putIfAbsent() method:

{code}
connection = connections.get(remoteId);
if (connection == null) {
  connection = new Connection(remoteId, serviceClass);
  Connection existing = connections.putIfAbsent(remoteId, connection);
  if (existing != null) {
    // ditch the one you just created as there is already a connection
    connection.close();
    connection = existing;
  }
}
{code}

> Replace guava Cache with ConcurrentHashMap for caching Connection in ipc Client
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-12475
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12475
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: conf, io, ipc
>            Reporter: Walter Su
>            Assignee: Walter Su
>         Attachments: HADOOP-12475.01.patch
>
>
> quote [~daryn] from HADOOP-11772:
> {quote}
> CacheBuilder is obscenely expensive for concurrent map, and it requires generating unnecessary
garbage even just to look up a key. Replace it with ConcurrentHashMap.
> I identified this issue that impaired my own perf testing under load. The slowdown isn't
just the sync. It's the expensive of Connection's ctor stalling other connections. The expensive
of ConnectionId#equals causes delays. Synch'ing on connections causes unfair contention unlike
a sync'ed method. Concurrency simply hides this.
> {quote}
> BTW, guava Cache is heavyweight. Per local test, ConcurrentHashMap has better overal
performance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message