jackrabbit-oak-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chetan Mehrotra <chetan.mehro...@gmail.com>
Subject Segment Store and unbounded instances of MappedByteBuffer causing high RAM usage
Date Tue, 19 Nov 2013 06:57:09 GMT

On some systems running on Oak (on Windows) where lots of nodes get
created we are seeing high RAM usage. The Java process memory (shown
directly under task manager. See note on Working Set below) remains
within expected range but system starts paging memory quite a bit and
slows down.

On checking the runtime state following things can be observed
* FileStore has 186 TarFile and each TarFile refers to 256MB
MappedByteBuffer taking around 46 GB of memory
* The JVM process constantly shows 50% CPU usage. Checking further
indictes that AsynchUpdate is running and Lucene indexing is being
* Windows Task Manager shows 7 .66 GB (total 8 GB) overall memory
usage with Java process only showing 1.4GB usage. No other process
account for so high usage
* Checking the Working Set [1] shows 7 GB memory being used by Java
process alone

>From my basic understanding of the Memory Mapped file usage it should
not cause msuch resource crunch and OS would be able to manage the
memory. However probably in Oak case all these TarFiles are getting
accessed frequently which leads to these memory mapped pages held in
physical memory.

Should we put upper cap on number of TarFiles opened?

If any other data needs to be collected to determine the problem cause
further then let me know?

Chetan Mehrotra
[1] http://msdn.microsoft.com/en-us/library/windows/desktop/cc441804(v=vs.85).aspx

View raw message