Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: <AANLkTim4sByxK-1FV9cnjLDPZ+8rW7oa8mjKVdD8GjbY@mail.gmail.com>
References: <AANLkTi=p3osjGXJ+LAFJhAHBF_aFGWjCso0Fz6ry+Naq@mail.gmail.com>
 <AANLkTim4sByxK-1FV9cnjLDPZ+8rW7oa8mjKVdD8GjbY@mail.gmail.com>
From: Todd Lipcon <todd@cloudera.com>
Date: Thu, 30 Dec 2010 07:38:31 -0800
Message-ID: <AANLkTim53LMABm4Kw61Zt8fHvW_bPogO3vZfOQPXyZ-E@mail.gmail.com>
Subject: Re: Java Commited Virtual Memory significally larged then Heap Memory
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=90e6ba53a4f4874e540498a27be7

--90e6ba53a4f4874e540498a27be7
Content-Type: text/plain; charset=ISO-8859-1

Hi Andrey,

Any chance you're using hadoop-lzo with CDH3b3? There was a leak in earlier
versions of hadoop-lzo that showed up under CDH3b3. You should upgrade to
the newest.

If that's not it, let me know, will keep thinking.

-Todd

On Thu, Dec 30, 2010 at 12:13 AM, Andrey Stepachev <octo47@gmail.com> wrote:

> Addition information:
>
> ps shows, that my HBase process eats up to 4GB of RSS.
>
> $ ps --sort=-rss -eopid,rss | head | grep HMaster
>  PID   RSS
> 23476 3824892
>
>
> 2010/12/30 Andrey Stepachev <octo47@gmail.com>
>
> > Hi All.
> >
> > After heavy load into hbase (single node, nondistributed test system) I
> got
> > 4Gb process size of my HBase java process.
> > On 6GB machine there was no room for anything else (disk cache and so
> on).
> >
> > Does anybody knows, what is going on, and how you solve this. What heap
> > memory is set on you hosts
> > and how much of RSS hbase process actually use.
> >
> > I don't see such things before, all tomcat and other java apps don't eats
> > significally more memory then -Xmx.
> >
> >  Connection name:   pid: 23476 org.apache.hadoop.hbase.master.HMaster
> > start   Virtual Machine:   Java HotSpot(TM) 64-Bit Server VM version
> > 17.1-b03   Vendor:   Sun Microsystems Inc.   Name:   23476@mars
>  Uptime:   12
> > hours 4 minutes   Process CPU time:   5 hours 45 minutes   JIT compiler:
>   HotSpot
> > 64-Bit Server Compiler   Total compile time:   19,223 seconds
> > ------------------------------
> >     Current heap size:     703 903 kbytes   Maximum heap size:   2 030
> 976kbytes    Committed memory:
> > 2 030 976 kbytes   Pending finalization:   0 objects      Garbage
> > collector:   Name = 'ParNew', Collections = 9 990, Total time spent = 5
> > minutes   Garbage collector:   Name = 'ConcurrentMarkSweep', Collections
> =
> > 20, Total time spent = 35,754 seconds
> > ------------------------------
> >     Operating System:   Linux 2.6.34.7-0.5-xen   Architecture:   amd64
>  Number of processors:
> > 8   Committed virtual memory:   4 403 512 kbytes     Total physical
> > memory:   6 815 744 kbytes   Free physical memory:      82 720 kbytes
>  Total swap space:
> > 8 393 924 kbytes   Free swap space:   8 050 880 kbytes
> >
> >
> >
> >
>


-- 
Todd Lipcon
Software Engineer, Cloudera

--90e6ba53a4f4874e540498a27be7--