Return-Path: X-Original-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3625710E61 for ; Mon, 9 Dec 2013 15:44:13 +0000 (UTC) Received: (qmail 83367 invoked by uid 500); 9 Dec 2013 15:44:11 -0000 Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org Received: (qmail 83326 invoked by uid 500); 9 Dec 2013 15:44:10 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-issues@hadoop.apache.org Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 83273 invoked by uid 99); 9 Dec 2013 15:44:09 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 09 Dec 2013 15:44:09 +0000 Date: Mon, 9 Dec 2013 15:44:09 +0000 (UTC) From: "Liang Xie (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HDFS-1605) Convert DFSInputStream synchronized sections to a ReadWrite lock MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HDFS-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843244#comment-13843244 ] Liang Xie commented on HDFS-1605: --------------------------------- Considering risk, i changes most of synchronized section with writeLock. only few places(like getFileLengh, some simple getter methods) use readLock in current impl, i think it's ok since both i and [~dhruba] observed the very same hotspot: getFileLengh, in similar scenario: HBase read. > Convert DFSInputStream synchronized sections to a ReadWrite lock > ---------------------------------------------------------------- > > Key: HDFS-1605 > URL: https://issues.apache.org/jira/browse/HDFS-1605 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs-client > Reporter: dhruba borthakur > Assignee: dhruba borthakur > Attachments: DFSClientRWlock.1.txt, DFSClientRWlock.3.txt, HDFS-1605.txt > > > Hbase does concurrent preads from multiple threads to different blocks of the same hdfs file. Each of these pread calls invoke DFSInputStream.getFileLength() and DFSInputStream.getBlockAt(). These methods are "synchronized", thus causing all the concurrent threads to serialize. It would help performance to convert this to a Read/Write lock -- This message was sent by Atlassian JIRA (v6.1.4#6159)