hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinitha Reddy Gankidi (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-11384) Add option for balancer to disperse getBlocks calls to avoid NameNode's rpc.CallQueueLength spike
Date Thu, 30 Mar 2017 20:36:41 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949757#comment-15949757
] 

Vinitha Reddy Gankidi edited comment on HDFS-11384 at 3/30/17 8:36 PM:
-----------------------------------------------------------------------

If we were to offload the calls to DN, dispersing calls wouldn't be a pressing issue. I would
like to get some feedback  on the various approaches discussed. [~benoyantony] [~daryn] [~liuml07]
[~zhaoyunjiong] I would love to hear your opinions.


was (Author: redvine):
If we were to offload the calls to DN, dispersing calls wouldn't be a pressing issue. I would
like to get some feedback  on the various approaches discussed. [~benoyantony] [~daryn] [~liuml07]
I would love to hear your opinions.

> Add option for balancer to disperse getBlocks calls to avoid NameNode's rpc.CallQueueLength
spike
> -------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-11384
>                 URL: https://issues.apache.org/jira/browse/HDFS-11384
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: balancer & mover
>    Affects Versions: 2.7.3
>            Reporter: yunjiong zhao
>            Assignee: yunjiong zhao
>         Attachments: balancer.day.png, balancer.week.png, HDFS-11384.001.patch, HDFS-11384.002.patch
>
>
> When running balancer on hadoop cluster which have more than 3000 Datanodes will cause
NameNode's rpc.CallQueueLength spike. We observed this situation could cause Hbase cluster
failure due to RegionServer's WAL timeout.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message