jackrabbit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ard Schrijvers (JIRA)" <j...@apache.org>
Subject [jira] Commented: (JCR-937) CacheManager max memory size
Date Wed, 24 Oct 2007 11:36:51 GMT

    [ https://issues.apache.org/jira/browse/JCR-937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12537289

Ard Schrijvers commented on JCR-937:

"Is the Lucene memory consumption due to the extremely long String properties, or has it to
do with the test setup? "

No, this is not because of long String properties, and yes, it is closely related to the test
setup. You are storing nodes of roughly 1 Mb, right? Now, depending on your SearchIndex configuration,
for example volatileIdleTime (how long before a memory index is flushed to disk) , bufferSize
(maximum number of documents that are held in a pending queue until added to the index), minMergeDocs
etc etc, it depends how large the memory lucene index will be (VolatileIndex).  I really think
the OOM is part of the way you have done the setup. If for example, you would replace the
1Mb String of 'aaaaa....' to a random piece of text of 1 Mb, the lucene indexing would take
much longer, hence the memory index wouldn't be able to grow that large, because volatileIdleTime
will be reached ealier and memory index will be persisted. 

> CacheManager max memory size
> ----------------------------
>                 Key: JCR-937
>                 URL: https://issues.apache.org/jira/browse/JCR-937
>             Project: Jackrabbit
>          Issue Type: Bug
>          Components: jackrabbit-core
>    Affects Versions: 1.3
>            Reporter: Xiaohua Lu
>            Assignee: Thomas Mueller
>            Priority: Minor
>         Attachments: CacheManagerTest.java
> I have ran into OutOfMemory a couple of times with Jackrabbit (cluster, 4 nodes, each
has 1G mem heap size). 
> After adding some debug into the CacheManager, I noticed that maxMemorySize (default
to 16M) is not really honored during resizeAll check.  Each individual MLRUItemStateCache
seems to honor the size, but the total number/size of MLRUItemStateCache is not. If you put
some print statement of totalMemoryUsed and unusedMemory, you can see that totalMemoryUsed
is more than 16M and unusedMemory is negative. 
> The other problem we have noticed during the profiling is there are a lots of other in
memory objects that are consuming memory but not included in CacheManager caches control.
One example is CachingHierarchyManager which consumed 58M out of 242M through its use of PathMap.
If CacheManager's maxSize can control the total cache size used by Jackrabbit, that would
be easier from a management's perspective. (btw, upper_limit in CachingHierarchyManager is
hardcoded and can't be control from outside)

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message