Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Thu, 21 Jan 2016 07:07:39 +0000 (UTC)
From: "Yu Li (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12932601.1453256878000.167938.1453360059939@Atlassian.JIRA>
In-Reply-To: <JIRA.12932601.1453256878000@Atlassian.JIRA>
References: <JIRA.12932601.1453256878000@Atlassian.JIRA>
 <JIRA.12932601.1453256878713@arcas>
Subject: [jira] [Commented] (HDFS-9666) Enable hdfs-client to read even
 remote SSD/RAM prior to local disk replica to improve random read
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-9666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15110188#comment-15110188 ] 

Yu Li commented on HDFS-9666:
-----------------------------

bq. However it looked like the benefits of reading from remote RAM were canceled by the RPC overhead, as compared to short-circuit reads from local disk
Agreed this is true for most *common* case. However, since SATA has much poor io performance than SSD/RAM, reading from remote SSD/RAM is useful to reduce spike in the system, or say it's good for reducing the Max latency rather than Avg. And since there's a switch to turn on/off the feature, user could choose to use it or not according to different scenarios.

> Enable hdfs-client to read even remote SSD/RAM prior to local disk replica to improve random read
> -------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-9666
>                 URL: https://issues.apache.org/jira/browse/HDFS-9666
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs-client
>    Affects Versions: 2.6.0, 2.7.0
>            Reporter: ade
>            Assignee: ade
>             Fix For: 2.7.2
>
>         Attachments: HDFS-9666.0.patch
>
>
> We want to improve random read performance of HDFS for HBase, so enabled the heterogeneous storage in our cluster. But there are only ~50% of datanode & regionserver hosts with SSD. we can set hfile with only ONE_SSD not ALL_SSD storagepolicy and the regionserver on none-SSD host can only read the local disk replica . So we developed this feature in hdfs client to read even remote SSD/RAM prior to local disk replica.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)