hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravi Prakash (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1605) Convert DFSInputStream synchronized sections to a ReadWrite lock
Date Fri, 08 May 2015 18:26:01 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14535173#comment-14535173

Ravi Prakash commented on HDFS-1605:

Thanks for the patch [~kartheek]
I'm afraid we may have changed the synchronization model for DFSInputStream in this patch.
For example, earlier 2 threads: one accessing getCurrentDatanode() and another seekToNewSource()
; would have synchronized on a single lock. However now, the two threads could proceed at
the same time and hence there's a race condition on currentNode. Is my understanding correct?

> Convert DFSInputStream synchronized sections to a ReadWrite lock
> ----------------------------------------------------------------
>                 Key: HDFS-1605
>                 URL: https://issues.apache.org/jira/browse/HDFS-1605
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs-client
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>              Labels: BB2015-05-RFC
>         Attachments: DFSClientRWlock.1.txt, DFSClientRWlock.3.txt, HADOOP-1605-trunk.patch,
> Hbase does concurrent preads from multiple threads to different blocks of the same hdfs
file. Each of these pread calls invoke DFSInputStream.getFileLength() and DFSInputStream.getBlockAt().
These methods are "synchronized", thus causing all the concurrent threads to serialize. It
would help performance to convert this to a Read/Write lock

This message was sent by Atlassian JIRA

View raw message