Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Wed, 3 Dec 2014 15:41:17 +0000 (UTC)
From: "Hudson (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12729078.1406085237000.68673.1417621277318@Atlassian.JIRA>
In-Reply-To: <JIRA.12729078.1406085237000@Atlassian.JIRA>
References: <JIRA.12729078.1406085237000@Atlassian.JIRA>
 <JIRA.12729078.1406085237634@arcas>
Subject: [jira] [Commented] (HDFS-6735) A minor optimization to avoid
 pread() be blocked by read() inside the same DFSInputStream
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14233116#comment-14233116 ] 

Hudson commented on HDFS-6735:
------------------------------

FAILURE: Integrated in Hadoop-Mapreduce-trunk #1978 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1978/])
HDFS-6735. A minor optimization to avoid pread() be blocked by read() inside the same DFSInputStream (Lars Hofhansl via stack) (stack: rev 7caa3bc98e6880f98c5c32c486a0c539f9fd3f5f)
* hadoop-hdfs-project/hadoop-hdfs/dev-support/findbugsExcludeFile.xml
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/shortcircuit/ShortCircuitReplica.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/LocatedBlocks.java
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/shortcircuit/ShortCircuitCache.java
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java


> A minor optimization to avoid pread() be blocked by read() inside the same DFSInputStream
> -----------------------------------------------------------------------------------------
>
>                 Key: HDFS-6735
>                 URL: https://issues.apache.org/jira/browse/HDFS-6735
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs-client
>    Affects Versions: 3.0.0
>            Reporter: Liang Xie
>            Assignee: Lars Hofhansl
>             Fix For: 2.7.0
>
>         Attachments: HDFS-6735-v2.txt, HDFS-6735-v3.txt, HDFS-6735-v4.txt, HDFS-6735-v5.txt, HDFS-6735-v6.txt, HDFS-6735-v7.txt, HDFS-6735-v8.txt, HDFS-6735.txt
>
>
> In current DFSInputStream impl, there're a couple of coarser-grained locks in read/pread path, and it has became a HBase read latency pain point so far. In HDFS-6698, i made a minor patch against the first encourtered lock, around getFileLength, in deed, after reading code and testing, it shows still other locks we could improve.
> In this jira, i'll make a patch against other locks, and a simple test case to show the issue and the improved result.
> This is important for HBase application, since in current HFile read path, we issue all read()/pread() requests in the same DFSInputStream for one HFile. (Multi streams solution is another story i had a plan to do, but probably will take more time than i expected)


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)