hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liang Xie (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1605) Convert DFSInputStream synchronized sections to a ReadWrite lock
Date Mon, 09 Dec 2013 15:44:09 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843244#comment-13843244
] 

Liang Xie commented on HDFS-1605:
---------------------------------

Considering risk, i changes most of synchronized section with writeLock. only few places(like
getFileLengh, some simple getter methods) use readLock in current impl,  i think it's ok since
both i and [~dhruba] observed the very same hotspot: getFileLengh, in similar scenario: HBase
read.

> Convert DFSInputStream synchronized sections to a ReadWrite lock
> ----------------------------------------------------------------
>
>                 Key: HDFS-1605
>                 URL: https://issues.apache.org/jira/browse/HDFS-1605
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs-client
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>         Attachments: DFSClientRWlock.1.txt, DFSClientRWlock.3.txt, HDFS-1605.txt
>
>
> Hbase does concurrent preads from multiple threads to different blocks of the same hdfs
file. Each of these pread calls invoke DFSInputStream.getFileLength() and DFSInputStream.getBlockAt().
These methods are "synchronized", thus causing all the concurrent threads to serialize. It
would help performance to convert this to a Read/Write lock



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Mime
View raw message