jackrabbit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Al Caponi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (JCR-1224) OutOfMemory on 64-Bit Linux
Date Wed, 05 Dec 2007 04:22:43 GMT

    [ https://issues.apache.org/jira/browse/JCR-1224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12548531
] 

Al Caponi commented on JCR-1224:
--------------------------------

Hi,
I also reproduced an OutOfMemory error with a test class similar to FirstHop.java. Here are
some findings.

Jackrabbit version: 1.3.3

Environment:
---------------------------
Windows XP Pro v.2002 SP2
AMD Athlon 64
1GB RAM

JRE: Sun JRE 1.5.0_11
---------------------------

Error message: java.lang.OutOfMemoryError: Java heap space

Stack from Eclipse debugger:
---------------------------
Thread [main] (Suspended (breakpoint at line 140 in IndexMerger))	
	IndexMerger.indexAdded(String, int) line: 140	
	MultiIndex$AddIndex.execute(MultiIndex) line: 1362	
	MultiIndex.executeAndLog(MultiIndex$Action) line: 872	
	MultiIndex.commitVolatileIndex() line: 926	
	MultiIndex.flush() line: 810	
	Recovery.run() line: 172	
	Recovery.run(MultiIndex, RedoLog) line: 85	
	MultiIndex.<init>(File, SearchIndex, ItemStateManager, NodeId, Set, NamespaceMappings)
line: 297	
	SearchIndex.doInit() line: 295	
	SearchIndex(AbstractQueryHandler).init(QueryHandlerContext) line: 44	
	SearchManager.initializeQueryHandler() line: 478	
	SearchManager.<init>(SearchConfig, NamespaceRegistryImpl, NodeTypeRegistry, ItemStateManager,
NodeId, SearchManager, NodeId) line: 231	
	RepositoryImpl$WorkspaceInfo.getSearchManager() line: 1580	
	RepositoryImpl.initWorkspace(RepositoryImpl$WorkspaceInfo) line: 570	
	RepositoryImpl.initStartupWorkspaces() line: 379	
	RepositoryImpl.<init>(RepositoryConfig) line: 286	
	RepositoryImpl.create(RepositoryConfig) line: 521	
	TransientRepository$2.getRepository() line: 245	
	TransientRepository.startRepository() line: 265	
	TransientRepository.login(Credentials, String) line: 333	
	TransientRepository.login() line: 388	
	JcrTest.main(String[]) line: 65	<< My test class
---------------------------

Findings:
Drilled down to an infinite loop in org.apache.jackrabbit.core.query.lucene.IndexMerger (line
140):
---------------------------
class IndexMerger extends Thread implements IndexListener {
...
    void indexAdded(String name, int numDocs) {
...
                while (upper < maxMergeDocs) {
                    indexBuckets.add(new IndexBucket(lower, upper, true));
                    lower = upper + 1;
                    upper *= mergeFactor;
...
                }
..
}
---------------------------

>From repository.xml <SearchIndex>
          <param name="minMergeDocs" value="100"/>
          <param name="maxMergeDocs" value="2147483647"/>
          <param name="mergeFactor" value="10"/>
The upper variable (init at 100) which grows by a factor of 10 constantly skips into negative
territory (e.g. 1215752192 --> -727379968) and would probably never equal to 2147483647
before running out of heap space.

<SearchIndex> from my repository.xml (copied from website sample and modified the persistence
manager to SimpleDbPersistenceManager (for MySQL))
---------------------------
...
        <SearchIndex class="org.apache.jackrabbit.core.query.lucene.SearchIndex">
            <param name="path" value="default/index"/>
            <param name="useCompoundFile" value="true"/>
            <param name="minMergeDocs" value="100"/>
            <param name="volatileIdleTime" value="3"/>
            <param name="maxMergeDocs" value="2147483647"/>
            <param name="mergeFactor" value="10"/>
            <param name="maxFieldLength" value="10000"/>
            <param name="bufferSize" value="10"/>
            <param name="cacheSize" value="1000"/>
            <param name="forceConsistencyCheck" value="false"/>
            <param name="enableConsistencyCheck" value="false"/>
            <param name="autoRepair" value="true"/>
            <param name="analyzer" value="org.apache.lucene.analysis.standard.StandardAnalyzer"/>
            <param name="queryClass" value="org.apache.jackrabbit.core.query.QueryImpl"/>
            <param name="respectDocumentOrder" value="true"/>
            <param name="resultFetchSize" value="2147483647"/>
            <param name="extractorPoolSize" value="0"/>
            <param name="extractorTimeout" value="100"/>
            <param name="extractorBackLogSize" value="100"/>
        </SearchIndex>
    </Workspace>
---------------------------

Changing the value of maxMergeDocs didn't help as in this particular stack trace, the value
of IndexMerger.maxMergeDocs was reset to 2147483647.

Regards,
Al


> OutOfMemory on 64-Bit Linux
> ---------------------------
>
>                 Key: JCR-1224
>                 URL: https://issues.apache.org/jira/browse/JCR-1224
>             Project: Jackrabbit
>          Issue Type: Bug
>          Components: indexing
>    Affects Versions: 1.4
>         Environment: Linux lysiosdev 2.6.23.1-21.fc7 #1 SMP Thu Nov 1 20:28:15 EDT 2007
x86_64 x86_64 x86_64 GNU/Linux
> java version "1.5.0_12"
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_12-b04)
> Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_12-b04, mixed mode)
>            Reporter: Andreas Julius
>
> On my 64-Bit environment OS/JVM I tried a "mvn clean install" and got an OutOfMemory
Exception.
> On my 32-Bit environment Mac OSX 10.5 Java 1.5 the tests were all  fine and the IndexMerger
was significant faster.
> Running org.apache.jackrabbit.test.TestAll
> 21.11.2007 10:29:51 *INFO * [IndexMerger] IndexMerger: merged 549 documents in 289 ms
into _a. (IndexMerger.java, line 304)
> 21.11.2007 10:29:55 *ERROR* [main] ImportHandler: fatal error encountered at line: 1,
column: 10 while parsing XML stream: org.xml.sax.SAXParseException: Attribute name "is" associated
with an element type "this" must be followed by the ' = ' character. (ImportHandler.java,
line 116)
> 21.11.2007 10:29:55 *ERROR* [main] ImportHandler: fatal error encountered at line: 1,
column: 10 while parsing XML stream: org.xml.sax.SAXParseException: Attribute name "is" associated
with an element type "this" must be followed by the ' = ' character. (ImportHandler.java,
line 104)
> 21.11.2007 10:29:59 *ERROR* [main] ImportHandler: fatal error encountered at line: -1,
column: -1 while parsing XML stream: org.xml.sax.SAXParseException: Premature end of file.
(ImportHandler.java, line 104)
> 21.11.2007 10:29:59 *ERROR* [main] ImportHandler: fatal error encountered at line: -1,
column: -1 while parsing XML stream: org.xml.sax.SAXParseException: Premature end of file.
(ImportHandler.java, line 116)
> 21.11.2007 10:30:45 *INFO * [IndexMerger] IndexMerger: merged 555 documents in 2015 ms
into _l. (IndexMerger.java, line 304)
> 21.11.2007 10:33:13 *INFO * [IndexMerger] IndexMerger: merged 412 documents in 25587
ms into _w. (IndexMerger.java, line 304)
> Exception in thread "Timer-1" java.lang.OutOfMemoryError: Java heap space
>         at org.apache.lucene.store.BufferedIndexOutput.<init>(BufferedIndexOutput.java:26)
>         at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:592)
>         at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:435)
>         at org.apache.lucene.util.BitVector.write(BitVector.java:122)
>         at org.apache.lucene.index.SegmentReader.doCommit(SegmentReader.java:236)
>         at org.apache.lucene.index.IndexReader.commit(IndexReader.java:794)
>         at org.apache.lucene.index.FilterIndexReader.doCommit(FilterIndexReader.java:190)
>         at org.apache.lucene.index.IndexReader.commit(IndexReader.java:825)
>         at org.apache.lucene.index.IndexReader.close(IndexReader.java:841)
>         at org.apache.jackrabbit.core.query.lucene.AbstractIndex.close(AbstractIndex.java:327)
>         at org.apache.jackrabbit.core.query.lucene.MultiIndex$DeleteIndex.execute(MultiIndex.java:1715)
>         at org.apache.jackrabbit.core.query.lucene.MultiIndex.executeAndLog(MultiIndex.java:936)
>         at org.apache.jackrabbit.core.query.lucene.MultiIndex.flush(MultiIndex.java:880)
>         at org.apache.jackrabbit.core.query.lucene.MultiIndex.checkFlush(MultiIndex.java:1110)
>         at org.apache.jackrabbit.core.query.lucene.MultiIndex.access$100(MultiIndex.java:75)
>         at org.apache.jackrabbit.core.query.lucene.MultiIndex$1.run(MultiIndex.java:324)
>         at java.util.TimerThread.mainLoop(Timer.java:512)
>         at java.util.TimerThread.run(Timer.java:462)
> 21.11.2007 10:34:37 *ERROR* [main] DatabasePersistenceManager: failed to write node state:
cfbffd6d-114d-4738-9383-48da2b5dbc1d (DatabasePersistenceManager.java, line 441)
> java.lang.OutOfMemoryError: Java heap space
>         at java.util.Properties$LineReader.<init>(Properties.java:346)
>         at java.util.Properties.load(Properties.java:284)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message