hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mafish Liu <maf...@gmail.com>
Subject Re: Datanode high memory usage
Date Tue, 01 Sep 2009 09:08:43 GMT
Both NameNode and DataNode will be affected by number of files greatly.
In my test, almost 60% memory are used in datanodes while storing 1m
files, and the value reach 80% with 2m files.
My test best is with 5 nodes, 1 namenode and 4 datanodes. All nodes
have 2GB memory and replication is 3.

2009/9/1 Stas Oskin <stas.oskin@gmail.com>:
> Hi.
>
> 2009/9/1 Mafish Liu <mafish@gmail.com>
>
>> Did you have many small files in your system?
>>
>>
> Yes, quite plenty.
>
> But this should influence the Namenode, and not the Datanode, correct?
>
> Regards.
>



-- 
Mafish@gmail.com

Mime
View raw message