hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brahma Reddy Battula <brahmareddy.batt...@huawei.com>
Subject RE: Namenode memory usage
Date Tue, 18 Jun 2013 01:45:59 GMT
Can you take heapdump and check ..? Here you can check which objects are using how much..



Command : jmap -histo:live namenodepid

________________________________
From: Personal [nitin.motgi@gmail.com]
Sent: Tuesday, June 18, 2013 7:20 AM
To: user@hadoop.apache.org
Subject: Re: Namenode memory usage


  E Lego



On Mon, Jun 17, 2013 at 1:04 PM, Patai Sangbutsarakum <silvianhadoop@gmail.com<mailto:silvianhadoop@gmail.com>>
wrote:

Hi Hadoopers,

My dedicated Namenode box has 48G of memory, 40G is allocated for NN. HEAP

This is from 50070/dfshealth.jsp
28540193 files and directories, 32324098 blocks = 60864291 total. Heap Size is 23.34 GB /
38.54 GB (60%)

The Heap is fluctuating between less than 20G up to almost 100%


However, from top command Residence size is constantly at 39G not matter how low of the memory
usage in defshealth.jsp page
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 4628 apps      19   0 40.4g  39g  22m S 132.9 83.9  44821:08 /usr/java/jdk/jre/bin/java -Dproc_namenode
-Xmx40000m


Is this the time to upgrade the ram to the namenode box?

I remember the easy rule of thumb is 150 bye of every 1M for blocks+file+dir, so 60864291
* 150byte is around 9G. I just don't understand why 40G seems to be used up.?
Please educate..

Hope this make sense
P


Mime
View raw message