hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Java Commited Virtual Memory significally larged then Heap Memory
Date Thu, 30 Dec 2010 15:38:31 GMT
Hi Andrey,

Any chance you're using hadoop-lzo with CDH3b3? There was a leak in earlier
versions of hadoop-lzo that showed up under CDH3b3. You should upgrade to
the newest.

If that's not it, let me know, will keep thinking.

-Todd

On Thu, Dec 30, 2010 at 12:13 AM, Andrey Stepachev <octo47@gmail.com> wrote:

> Addition information:
>
> ps shows, that my HBase process eats up to 4GB of RSS.
>
> $ ps --sort=-rss -eopid,rss | head | grep HMaster
>  PID   RSS
> 23476 3824892
>
>
> 2010/12/30 Andrey Stepachev <octo47@gmail.com>
>
> > Hi All.
> >
> > After heavy load into hbase (single node, nondistributed test system) I
> got
> > 4Gb process size of my HBase java process.
> > On 6GB machine there was no room for anything else (disk cache and so
> on).
> >
> > Does anybody knows, what is going on, and how you solve this. What heap
> > memory is set on you hosts
> > and how much of RSS hbase process actually use.
> >
> > I don't see such things before, all tomcat and other java apps don't eats
> > significally more memory then -Xmx.
> >
> >  Connection name:   pid: 23476 org.apache.hadoop.hbase.master.HMaster
> > start   Virtual Machine:   Java HotSpot(TM) 64-Bit Server VM version
> > 17.1-b03   Vendor:   Sun Microsystems Inc.   Name:   23476@mars
>  Uptime:   12
> > hours 4 minutes   Process CPU time:   5 hours 45 minutes   JIT compiler:
>   HotSpot
> > 64-Bit Server Compiler   Total compile time:   19,223 seconds
> > ------------------------------
> >     Current heap size:     703 903 kbytes   Maximum heap size:   2 030
> 976kbytes    Committed memory:
> > 2 030 976 kbytes   Pending finalization:   0 objects      Garbage
> > collector:   Name = 'ParNew', Collections = 9 990, Total time spent = 5
> > minutes   Garbage collector:   Name = 'ConcurrentMarkSweep', Collections
> =
> > 20, Total time spent = 35,754 seconds
> > ------------------------------
> >     Operating System:   Linux 2.6.34.7-0.5-xen   Architecture:   amd64
>  Number of processors:
> > 8   Committed virtual memory:   4 403 512 kbytes     Total physical
> > memory:   6 815 744 kbytes   Free physical memory:      82 720 kbytes
>  Total swap space:
> > 8 393 924 kbytes   Free swap space:   8 050 880 kbytes
> >
> >
> >
> >
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message