hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5644) Reduce lock contention in libhdfs.
Date Mon, 09 Dec 2013 18:50:08 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843396#comment-13843396
] 

Colin Patrick McCabe commented on HDFS-5644:
--------------------------------------------

libhdfs doesn't actually use locking to get the JNI environment.  It uses thread-local data
for that, since you need a different {{JNIEnv}} per thread.

The shared hash table stores global class references.  Since this is insert-only (we never
remove elements), and we should not need to resize this if we choose the initial size properly,
it would be relatively easy to create a concurrent version of this.

> Reduce lock contention in libhdfs.
> ----------------------------------
>
>                 Key: HDFS-5644
>                 URL: https://issues.apache.org/jira/browse/HDFS-5644
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: libhdfs
>    Affects Versions: 3.0.0, 2.2.0
>            Reporter: Chris Nauroth
>
> libhdfs uses locking internally for coordinating access to shared hash tables and the
JNI environment.  The scope of this issue is to improve performance of libhdfs by reducing
lock contention.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Mime
View raw message