hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zhangduo (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5270) Use thread pools in the datenode daemons
Date Sat, 09 May 2015 01:45:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14536103#comment-14536103
] 

zhangduo commented on HDFS-5270:
--------------------------------

Yeah HTTP/2 is great, but netty also uses a thread pool, so the problem is still there unless
we rewrite all the pipeline related logic which means we still need to keep backward compatibility...

Anyway, I will work on a patch which at least can pass unit tests. And I'm also open to HDFS-7966.
But I do not think there is a way which we do not need to eat this shit if the shit is like
a big mountain in front of us...:(

> Use thread pools in the datenode daemons
> ----------------------------------------
>
>                 Key: HDFS-5270
>                 URL: https://issues.apache.org/jira/browse/HDFS-5270
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>            Reporter: Haohui Mai
>            Assignee: Haohui Mai
>              Labels: BB2015-05-TBR
>         Attachments: HDFS-5270.000.patch, HDFS-5270.2.patch, TestConcurrentAccess.java
>
>
> The current implementation of the datanode creates a thread when a new request comes
in. This incurs high overheads for the creation / destruction of threads, making the datanode
unstable under high concurrent loads.
> This JIRA proposes to use a thread pool to reduce the overheads.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message