hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-11898) DFSClient#isHedgedReadsEnabled() should be per client flag
Date Thu, 01 Jun 2017 00:37:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16032260#comment-16032260
] 

Konstantin Shvachko edited comment on HDFS-11898 at 6/1/17 12:36 AM:
---------------------------------------------------------------------

The concept of {{HEDGED_READ_THREAD_POOL}} looks broken to me.
* I think we should not have it static and name it up to naming conventions: {{hedgedReadThreadPool}}.
* DFSClients are cached so this will be a common pool per URI. Keeping it common for all URIs
is not right.
* Also I think we should reset {{hedgedReadThreadPool}} in {{initThreadsNumForHedgedReads()}}
when the number of threads is {{num <= 0}}. Now we reset the pool if {{num > 0}}, why
0 should be different.
* This should solve HDFS-11900 as well.
* Might as well remove unused variable {{smallBufferSize}}


was (Author: shv):
The concept of {{HEDGED_READ_THREAD_POOL}} looks broken to me.
* I think we should not have it static and name it up to naming conventions: {{hedgedReadThreadPool}}.
* DFSClients are cached so this will be a common pool per URI. Keeping it common for all URIs
is not right.
* Also I think we should reset {{hedgedReadThreadPool}} in {{initThreadsNumForHedgedReads()}}
when the number of threads is {{num <= 0}}. Now we reset the pool if {{num > 0}}, why
0 should be different.
* This should solve HDFS-11900 as well.

> DFSClient#isHedgedReadsEnabled() should be per client flag 
> -----------------------------------------------------------
>
>                 Key: HDFS-11898
>                 URL: https://issues.apache.org/jira/browse/HDFS-11898
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs-client
>            Reporter: Vinayakumar B
>            Assignee: Vinayakumar B
>         Attachments: HDFS-11898-01.patch
>
>
> DFSClient#isHedgedReadsEnabled() returns value based on static {{HEDGED_READ_THREAD_POOL}}.

> Hence if any of the client initialized this in JVM, all remaining client reads will be
going through hedged read itself.
> This flag should be per client value.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message