Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Tue, 1 Feb 2011 23:03:29 +0000 (UTC)
From: "Todd Lipcon (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: 
 <1154863613.3951.1296601409111.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: <9380293.291961296374744953.JavaMail.jira@thor>
Subject: [jira] Commented: (HDFS-1605) Convert DFSInputStream synchronized
 sections to a ReadWrite lock
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12989438#comment-12989438 ] 

Todd Lipcon commented on HDFS-1605:
-----------------------------------

Hey Dhruba. I've never seen this lock actually be a problem in practice given current performance issues in DFSInputStream. I understand from Jonathan Gray that you're also testing some other patches to improve DFSInputStream performance. Maybe it would make sense to put those upstream first?

(or do you have some benchmarks that shows that the rwlock helps things even with an otherwise "stock" DFSInputStream)?

> Convert DFSInputStream synchronized sections to a ReadWrite lock
> ----------------------------------------------------------------
>
>                 Key: HDFS-1605
>                 URL: https://issues.apache.org/jira/browse/HDFS-1605
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs client
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>         Attachments: DFSClientRWlock.1.txt, DFSClientRWlock.3.txt
>
>
> Hbase does concurrent preads from multiple threads to different blocks of the same hdfs file. Each of these pread calls invoke DFSInputStream.getFileLength() and DFSInputStream.getBlockAt(). These methods are "synchronized", thus causing all the concurrent threads to serialize. It would help performance to convert this to a Read/Write lock

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira