hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5270) Use thread pools in the datenode daemons
Date Tue, 19 May 2015 19:01:03 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14551023#comment-14551023
] 

Colin Patrick McCabe commented on HDFS-5270:
--------------------------------------------

Thanks for looking at this, [~Apache9].

I think this could be done in a much simpler way by just using the existing types we have
({{DataXceiver, DataXceiverServer}}) without adding new Stopper types.  There are a few places
in the code where we reference {{Thread}}-- we could change those to reference {{DataXceiver}}
directly.  Then we could have a method in {{DataXceiver}} that puts the thread back into the
thread pool of the {{DataXceiverServer}}.

There are also a bunch of changes that are unrelated, like whitespace changes or changing
"while true" to "for (;;)" that we should avoid to minimize the size of the patch.

> Use thread pools in the datenode daemons
> ----------------------------------------
>
>                 Key: HDFS-5270
>                 URL: https://issues.apache.org/jira/browse/HDFS-5270
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>            Reporter: Haohui Mai
>            Assignee: zhangduo
>              Labels: BB2015-05-TBR
>         Attachments: HDFS-5270.000.patch, HDFS-5270.2.patch, HDFS-5270.3.patch, HDFS-5270.4.patch,
TestConcurrentAccess.java
>
>
> The current implementation of the datanode creates a thread when a new request comes
in. This incurs high overheads for the creation / destruction of threads, making the datanode
unstable under high concurrent loads.
> This JIRA proposes to use a thread pool to reduce the overheads.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message