hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Haohui Mai <h...@hortonworks.com>
Subject Re: datanode is slow
Date Thu, 20 Feb 2014 18:42:41 GMT
It looks like your datanode is overloaded. You can scale your system by
adding more datanodes.

You can also try tighten the admission control to recover. You can lower
the number of dfs.datanode.max.transfer.threads so that the datanode
accepts fewer concurrent requests (but which also means that it serves less
number of clients).

~Haohui



On Thu, Feb 20, 2014 at 8:44 AM, lei liu <liulei412@gmail.com> wrote:

> I use Hbase0.94 and CDH4. There are 25729 tcp connections in one
> machine,example:
> hadoop@apayhbs081 ~ $ netstat -a | wc -l
> 25729
>
> The linux configration is :
>        soft    core            0
>        hard    rss             10000
>        hard    nproc           20
>        soft    nproc           20
>        hard    nproc           50
>        hard    nproc           0
>        maxlogins       4
>        nproc          20480
>         nofile         204800
>
>
> When there are 25729 tcp connections in one machine, the datanode is very
> slow.
> How can I resolve the question?
>
>
>
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
View raw message