hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hairong Kuang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4116) Balancer should provide better resource management
Date Tue, 23 Sep 2008 18:03:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12633828#action_12633828
] 

Hairong Kuang commented on HADOOP-4116:
---------------------------------------

As I said, my intention is that receiveResponse never times out in normal state no matter
how slow the other side is. Setting KeepAlive is for detecting the other side's machine gets
crashed suddenly so it won't wait there forever. But for all other cases, it will return eventually.
Does it make sense?

> Balancer should provide better resource management
> --------------------------------------------------
>
>                 Key: HADOOP-4116
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4116
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.17.0
>            Reporter: Raghu Angadi
>            Assignee: Hairong Kuang
>            Priority: Blocker
>             Fix For: 0.18.2, 0.19.0
>
>         Attachments: balancerRM.patch, balancerRM1.patch
>
>
> The number of threads are currently limited on datanodes. Once these threads are occupied,
DataNode does not accept any more requests (DOS). Recently we saw a case where most of the
256 threads were waiting in {{DataXceiver.replaceBlock()}} trying to acquire  {{balancingSem}}.
 Since rebalancing  is (heavily) throttled, I would think this would be the common case. 
> These operations waiting  for active rebalancing threads to finish need not take up a
thread. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message