Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 00E881035E for ; Tue, 18 Jun 2013 01:46:37 +0000 (UTC) Received: (qmail 37887 invoked by uid 500); 18 Jun 2013 01:46:32 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 37615 invoked by uid 500); 18 Jun 2013 01:46:32 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 37608 invoked by uid 99); 18 Jun 2013 01:46:32 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 18 Jun 2013 01:46:32 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of brahmareddy.battula@huawei.com designates 119.145.14.65 as permitted sender) Received: from [119.145.14.65] (HELO szxga02-in.huawei.com) (119.145.14.65) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 18 Jun 2013 01:46:25 +0000 Received: from 172.24.2.119 (EHLO szxeml213-edg.china.huawei.com) ([172.24.2.119]) by szxrg02-dlp.huawei.com (MOS 4.3.4-GA FastPath queued) with ESMTP id BDL07021; Tue, 18 Jun 2013 09:46:01 +0800 (CST) Received: from SZXEML460-HUB.china.huawei.com (10.82.67.203) by szxeml213-edg.china.huawei.com (172.24.2.30) with Microsoft SMTP Server (TLS) id 14.1.323.7; Tue, 18 Jun 2013 09:45:59 +0800 Received: from SZXEML510-MBX.china.huawei.com ([169.254.3.176]) by szxeml460-hub.china.huawei.com ([10.82.67.203]) with mapi id 14.01.0323.007; Tue, 18 Jun 2013 09:46:00 +0800 From: Brahma Reddy Battula To: "user@hadoop.apache.org" Subject: RE: Namenode memory usage Thread-Topic: Namenode memory usage Thread-Index: AQHOa5Xc1rIsfruEc0aOgqTcyMy20Zk6BTGAgACttSs= Date: Tue, 18 Jun 2013 01:45:59 +0000 Message-ID: <8AD4EE147886274A8B495D6AF407DF6915E962AE@szxeml510-mbx.china.huawei.com> References: ,<1371511220558.ab833d4@Nodemailer> In-Reply-To: <1371511220558.ab833d4@Nodemailer> Accept-Language: en-US, zh-CN Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.18.96.93] Content-Type: multipart/alternative; boundary="_000_8AD4EE147886274A8B495D6AF407DF6915E962AEszxeml510mbxchi_" MIME-Version: 1.0 X-CFilter-Loop: Reflected X-Virus-Checked: Checked by ClamAV on apache.org --_000_8AD4EE147886274A8B495D6AF407DF6915E962AEszxeml510mbxchi_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Can you take heapdump and check ..? Here you can check which objects are us= ing how much.. Command : jmap -histo:live namenodepid ________________________________ From: Personal [nitin.motgi@gmail.com] Sent: Tuesday, June 18, 2013 7:20 AM To: user@hadoop.apache.org Subject: Re: Namenode memory usage E Lego On Mon, Jun 17, 2013 at 1:04 PM, Patai Sangbutsarakum > wrote: Hi Hadoopers, My dedicated Namenode box has 48G of memory, 40G is allocated for NN. HEAP This is from 50070/dfshealth.jsp 28540193 files and directories, 32324098 blocks =3D 60864291 total. Heap Si= ze is 23.34 GB / 38.54 GB (60%) The Heap is fluctuating between less than 20G up to almost 100% However, from top command Residence size is constantly at 39G not matter ho= w low of the memory usage in defshealth.jsp page PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4628 apps 19 0 40.4g 39g 22m S 132.9 83.9 44821:08 /usr/java/jdk= /jre/bin/java -Dproc_namenode -Xmx40000m Is this the time to upgrade the ram to the namenode box? I remember the easy rule of thumb is 150 bye of every 1M for blocks+file+di= r, so 60864291 * 150byte is around 9G. I just don't understand why 40G seem= s to be used up.? Please educate.. Hope this make sense P --_000_8AD4EE147886274A8B495D6AF407DF6915E962AEszxeml510mbxchi_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Can you take heapdump and check ..? Here you can check which objects are= using how much..

 

Command : jmap -histo:live namenodepid

From: Personal [nitin.motgi@gmail.com]
Sent: Tuesday, June 18, 2013 7:20 AM
To: user@hadoop.apache.org
Subject: Re: Namenode memory usage

  
  E Lego


On Mon, Jun 17, 2013 at 1:04 PM, Patai Sangbutsarakum = <silvianhad= oop@gmail.com> wrote:

Hi Hadoopers,

My dedicated Namenode box has 48G of memory, 40G is allocated for NN. = HEAP

This is from 50070/dfshealth.jsp
28540193 files and directories, 32324098 blocks =3D 6= 0864291 total. Heap Size is 23.34 GB / 38.54 GB (60%) 

The Heap is fluctuating between less than 20G up to a= lmost 100%


However, from top command Residence size is constantly at 39G not matt= er how low of the memory usage in defshealth.jsp page
  PID USER      PR  NI  VIRT  RES &= nbsp;SHR S %CPU %MEM    TIME+  COMMAND     &nb= sp;                     &= nbsp;                    =                     &nbs= p;           
 4628 apps      19   0 40.4g  39g  = 22m S 132.9 83.9  44821:08 /usr/java/jdk/jre/bin/java -Dproc_namenode = -Xmx40000m 


Is this the time to upgrade the ram to the namenode box?

I remember the easy rule of thumb is 150 bye of every 1M for blocks= 3;file+dir, so 60864291 * 150byte is around 9G. I= just don't understand why 40G seems to be used up.?
Please educate..

Hope this make sense
P

--_000_8AD4EE147886274A8B495D6AF407DF6915E962AEszxeml510mbxchi_--