hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "bijaya (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-8955) Support 'hedged' write in DFSClient
Date Wed, 26 Aug 2015 03:46:46 GMT
bijaya created HDFS-8955:

             Summary: Support 'hedged' write in DFSClient
                 Key: HDFS-8955
                 URL: https://issues.apache.org/jira/browse/HDFS-8955
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: hdfs-client
    Affects Versions: 3.0.0
            Reporter: bijaya
            Assignee: Liang Xie
             Fix For: 2.4.0

This is a placeholder of hdfs related stuff backport from https://issues.apache.org/jira/browse/HBASE-7509

The quorum read ability should be helpful especially to optimize read outliers

we can utilize "dfs.dfsclient.quorum.read.threshold.millis" & "dfs.dfsclient.quorum.read.threadpool.size"
to enable/disable the hedged read ability from client side(e.g. HBase), and by using DFSQuorumReadMetrics,
we could export the interested metric valus into client system(e.g. HBase's regionserver metric).

The core logic is in pread code path, we decide to goto the original fetchBlockByteRange or
the new introduced fetchBlockByteRangeSpeculative per the above config items.

This message was sent by Atlassian JIRA

View raw message