hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liang Xie (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5776) Support 'hedged' reads in DFSClient
Date Tue, 21 Jan 2014 04:37:20 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877177#comment-13877177
] 

Liang Xie commented on HDFS-5776:
---------------------------------

bq. Shut down this executor in the finally? Don't want it sticking around.
OK, add "executor.shutdown()" now, but i didn't put it into finally block, since no exception
within it and i don't like to move the executor definition before the try block:)

bq. Your nice new metrics showed in your above test? They made sense (I suppose they must
basically work since your test relies on them).
the new metrics also useful for HBase side, in our HBase related change, we could gather these
new metrics into RegionServerMetrics or some other place:)

bq. s/Throw/Throwing/
done, thanks.

{quote}
Needs a space between ie and errMsg?
+ + " from any node: " + ie + errMsg
{quote}
good catch, added a space inside getBestNodeErrorString now


> Support 'hedged' reads in DFSClient
> -----------------------------------
>
>                 Key: HDFS-5776
>                 URL: https://issues.apache.org/jira/browse/HDFS-5776
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs-client
>    Affects Versions: 3.0.0
>            Reporter: Liang Xie
>            Assignee: Liang Xie
>         Attachments: HDFS-5776-v2.txt, HDFS-5776-v3.txt, HDFS-5776.txt
>
>
> This is a placeholder of hdfs related stuff backport from https://issues.apache.org/jira/browse/HBASE-7509
> The quorum read ability should be helpful especially to optimize read outliers
> we can utilize "dfs.dfsclient.quorum.read.threshold.millis" & "dfs.dfsclient.quorum.read.threadpool.size"
to enable/disable the hedged read ability from client side(e.g. HBase), and by using DFSQuorumReadMetrics,
we could export the interested metric valus into client system(e.g. HBase's regionserver metric).
> The core logic is in pread code path, we decide to goto the original fetchBlockByteRange
or the new introduced fetchBlockByteRangeSpeculative per the above config items.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message