jackrabbit-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Johannes Utzig <m...@jutzig.de>
Subject Out Of Memory Error while indexing
Date Mon, 08 Feb 2010 19:18:13 GMT
Hi,

I am using Jackrabbit as a version control system for files (like 
CVS/SVN). I was doing some testing with large binary files and keep 
getting an out of memory error.
This is most likely a configuration mistake on my side, but I can’t seem 
to figure out what I’m doing wrong so hopefully someone is able to point 
me into the right direction.
The file I’m trying to commit (as the jcr:data Property of an 
nt:resource) is an iso image of about 700MB.
As the jcr:mimeType I used application/octet-stream in the hope that 
this would prevent Lucene from indexing the file (I don’t want files 
that large to be indexed). I would prefer to not completely disable 
indexing. Is there a way to prevent Jackrabbit from indexing 
extraordinary large properties?
As for the environment, I use Jackrabbit 1.6.0. The jackrabbit webapp is 
deployed in a tomcat and the clients connect with RMI.
The heap for tomcat is set to 512m
When I try to commit this file, both client and tomcat heap space 
consumption stay pretty low for most of the time. Then (I guess when all 
the contents have been transferred) the tomcat heap rapidly increases 
until I get an out of memory error with the stacktrace provided further 
down.
My repository.xml is the default one except for a DataStorage.
I was hoping that since I did not define any TextExtractors, that 
Jackrabbit would not try to index this huge binary property, but that’s 
apparently where it crashes.

Thanks in advance for any pointer.

Best regards,
Johannes

Repository Config:


<!DOCTYPE Repository PUBLIC "-//The Apache Software Foundation//DTD 
Jackrabbit 1.6//EN"
"http://jackrabbit.apache.org/dtd/repository-1.6.dtd">
<Repository>

<FileSystem class="org.apache.jackrabbit.core.fs.local.LocalFileSystem">
<param name="path" value="${rep.home}/repository"/>
</FileSystem>

<Security appName="Jackrabbit">

<SecurityManager 
class="org.apache.jackrabbit.core.security.simple.SimpleSecurityManager" 
workspaceName="security">
</SecurityManager>
<AccessManager 
class="org.apache.jackrabbit.core.security.simple.SimpleAccessManager">
</AccessManager>

<LoginModule 
class="org.apache.jackrabbit.core.security.simple.SimpleLoginModule">
<param name="anonymousId" value="anonymous"/>
<param name="adminId" value="admin"/>
</LoginModule>
</Security>

<Workspaces rootPath="${rep.home}/workspaces" defaultWorkspace="default"/>
<Workspace name="${wsp.name}">
<FileSystem class="org.apache.jackrabbit.core.fs.local.LocalFileSystem">
<param name="path" value="${wsp.home}"/>
</FileSystem>
<PersistenceManager 
class="org.apache.jackrabbit.core.persistence.bundle.DerbyPersistenceManager">
<param name="url" value="jdbc:derby:${wsp.home}/db;create=true"/>
<param name="schemaObjectPrefix" value="${wsp.name}_"/>
</PersistenceManager>
<SearchIndex class="org.apache.jackrabbit.core.query.lucene.SearchIndex">
<param name="path" value="${wsp.home}/index"/>
<param name="extractorPoolSize" value="2"/>
<param name="supportHighlighting" value="true"/>
</SearchIndex>
</Workspace>

<Versioning rootPath="${rep.home}/version">
<FileSystem class="org.apache.jackrabbit.core.fs.local.LocalFileSystem">
<param name="path" value="${rep.home}/version" />
</FileSystem>
<PersistenceManager 
class="org.apache.jackrabbit.core.persistence.bundle.DerbyPersistenceManager">
<param name="url" value="jdbc:derby:${rep.home}/version/db;create=true"/>
<param name="schemaObjectPrefix" value="version_"/>
</PersistenceManager>
</Versioning>
<SearchIndex class="org.apache.jackrabbit.core.query.lucene.SearchIndex">
<param name="path" value="${rep.home}/repository/index"/>
<param name="extractorPoolSize" value="2"/>
<param name="supportHighlighting" value="true"/>
</SearchIndex>

<DataStore class="org.apache.jackrabbit.core.data.FileDataStore">
<param name="path" value="${rep.home}/repository/datastore"/>
<param name="minRecordLength" value="100"/>
</DataStore>
</Repository>


Stacktrace:

05.02.2010 14:48:16 *WARN * LazyTextExtractorField: Exception reading 
value for field: Stream closed (LazyText
ExtractorField.java, line 94)
Exception in thread "Timer-1" java.lang.IllegalStateException: this 
writer hit an OutOfMemoryError; cannot com
mit
at org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:3353)
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3408)
at 
org.apache.jackrabbit.core.query.lucene.AbstractIndex.commit(AbstractIndex.java:363)
at 
org.apache.jackrabbit.core.query.lucene.VolatileIndex.commit(VolatileIndex.java:141)
at 
org.apache.jackrabbit.core.query.lucene.PersistentIndex.copyIndex(PersistentIndex.java:105)
at 
org.apache.jackrabbit.core.query.lucene.MultiIndex$VolatileCommit.execute(MultiIndex.java:1984)
at 
org.apache.jackrabbit.core.query.lucene.MultiIndex.executeAndLog(MultiIndex.java:1000)
at 
org.apache.jackrabbit.core.query.lucene.MultiIndex.commitVolatileIndex(MultiIndex.java:1048)
at 
org.apache.jackrabbit.core.query.lucene.MultiIndex.flush(MultiIndex.java:893)
at 
org.apache.jackrabbit.core.query.lucene.MultiIndex.checkFlush(MultiIndex.java:1164)
at 
org.apache.jackrabbit.core.query.lucene.MultiIndex.access$100(MultiIndex.java:80)
at 
org.apache.jackrabbit.core.query.lucene.MultiIndex$1.run(MultiIndex.java:317)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
05.02.2010 14:48:16 *ERROR* SearchManager: Error indexing node. 
(SearchManager.java, line 490)
java.io.IOException: Java heap space
at 
org.apache.jackrabbit.core.query.lucene.Util.createIOException(Util.java:114)
at 
org.apache.jackrabbit.core.query.lucene.AbstractIndex.addDocuments(AbstractIndex.java:199)
at 
org.apache.jackrabbit.core.query.lucene.VolatileIndex.commitPending(VolatileIndex.java:171)
at 
org.apache.jackrabbit.core.query.lucene.VolatileIndex.addDocuments(VolatileIndex.java:82)
at 
org.apache.jackrabbit.core.query.lucene.MultiIndex$AddNode.execute(MultiIndex.java:1599)
at 
org.apache.jackrabbit.core.query.lucene.MultiIndex.executeAndLog(MultiIndex.java:1000)
at 
org.apache.jackrabbit.core.query.lucene.MultiIndex.update(MultiIndex.java:429)
at 
org.apache.jackrabbit.core.query.lucene.SearchIndex.updateNodes(SearchIndex.java:588)
at org.apache.jackrabbit.core.SearchManager.onEvent(SearchManager.java:486)
at 
org.apache.jackrabbit.core.observation.EventConsumer.consumeEvents(EventConsumer.java:244)
at 
org.apache.jackrabbit.core.observation.ObservationDispatcher.dispatchEvents(ObservationDispatcher.j
ava:201)
at 
org.apache.jackrabbit.core.observation.EventStateCollection.dispatch(EventStateCollection.java:464)


at 
org.apache.jackrabbit.core.observation.DelegatingObservationDispatcher.dispatch(DelegatingObservati
onDispatcher.java:127)
at 
org.apache.jackrabbit.core.observation.DelegatingObservationDispatcher.dispatchEvents(DelegatingObs
ervationDispatcher.java:99)
at 
org.apache.jackrabbit.core.observation.EventStateCollection.dispatch(EventStateCollection.java:464)


at 
org.apache.jackrabbit.core.state.SharedItemStateManager$Update.end(SharedItemStateManager.java:760)


at 
org.apache.jackrabbit.core.state.SharedItemStateManager.update(SharedItemStateManager.java:1115)
at 
org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemStateManager.java:351)
at 
org.apache.jackrabbit.core.state.LocalItemStateManager.update(LocalItemStateManager.java:326)
at 
org.apache.jackrabbit.core.version.AbstractVersionManager$WriteOperation.save(AbstractVersionManage
r.java:189)
at 
org.apache.jackrabbit.core.version.AbstractVersionManager.checkin(AbstractVersionManager.java:442)
at 
org.apache.jackrabbit.core.version.VersionManagerImpl$2.run(VersionManagerImpl.java:290)
at 
org.apache.jackrabbit.core.version.VersionManagerImpl$DynamicESCFactory.doSourced(VersionManagerImp
l.java:586)
at 
org.apache.jackrabbit.core.version.VersionManagerImpl.checkin(VersionManagerImpl.java:281)
at 
org.apache.jackrabbit.core.version.XAVersionManager.checkin(XAVersionManager.java:180)
at org.apache.jackrabbit.core.NodeImpl.checkin(NodeImpl.java:3367)
at org.apache.jackrabbit.core.NodeImpl.checkin(NodeImpl.java:3346)
at org.apache.jackrabbit.rmi.server.ServerNode.checkin(ServerNode.java:335)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
at sun.rmi.transport.Transport$1.run(Transport.java:159)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
at 
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2882)
at 
java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:515)
at java.lang.StringBuffer.append(StringBuffer.java:306)
at 
org.apache.jackrabbit.core.query.lucene.LazyTextExtractorField.stringValue(LazyTextExtractorField.j
ava:91)
at 
org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:109)
at 
org.apache.lucene.index.DocFieldConsumersPerField.processFields(DocFieldConsumersPerField.java:36)
at 
org.apache.lucene.index.DocFieldProcessorPerThread.processDocument(DocFieldProcessorPerThread.java:
234)
at 
org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:765)
at 
org.apache.lucene.index.DocumentsWriter.addDocument(DocumentsWriter.java:743)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1917)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1895)
at 
org.apache.jackrabbit.core.query.lucene.AbstractIndex$1.call(AbstractIndex.java:183)
at 
org.apache.jackrabbit.core.query.lucene.DynamicPooledExecutor$1.call(DynamicPooledExecutor.java:109
)
at EDU.oswego.cs.dl.util.concurrent.FutureResult$1.run(Unknown Source)
at 
EDU.oswego.cs.dl.util.concurrent.PooledExecutor$RunWhenBlocked.blockedAction(Unknown 
Source)
at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.execute(Unknown Source)
at 
org.apache.jackrabbit.core.query.lucene.DynamicPooledExecutor.executeAndWait(DynamicPooledExecutor.
java:113)
at 
org.apache.jackrabbit.core.query.lucene.AbstractIndex.addDocuments(AbstractIndex.java:188)
at 
org.apache.jackrabbit.core.query.lucene.VolatileIndex.commitPending(VolatileIndex.java:171)
at 
org.apache.jackrabbit.core.query.lucene.VolatileIndex.addDocuments(VolatileIndex.java:82)
at 
org.apache.jackrabbit.core.query.lucene.MultiIndex$AddNode.execute(MultiIndex.java:1599)
at 
org.apache.jackrabbit.core.query.lucene.MultiIndex.executeAndLog(MultiIndex.java:1000)
at 
org.apache.jackrabbit.core.query.lucene.MultiIndex.update(MultiIndex.java:429)
at 
org.apache.jackrabbit.core.query.lucene.SearchIndex.updateNodes(SearchIndex.java:588)
at org.apache.jackrabbit.core.SearchManager.onEvent(SearchManager.java:486)
at 
org.apache.jackrabbit.core.observation.EventConsumer.consumeEvents(EventConsumer.java:244)
at 
org.apache.jackrabbit.core.observation.ObservationDispatcher.dispatchEvents(ObservationDispatcher.j
ava:201)
at 
org.apache.jackrabbit.core.observation.EventStateCollection.dispatch(EventStateCollection.java:464)


at 
org.apache.jackrabbit.core.observation.DelegatingObservationDispatcher.dispatch(DelegatingObservati
onDispatcher.java:127)
at 
org.apache.jackrabbit.core.observation.DelegatingObservationDispatcher.dispatchEvents(DelegatingObs
ervationDispatcher.java:99)
at 
org.apache.jackrabbit.core.observation.EventStateCollection.dispatch(EventStateCollection.java:464)


Mime
View raw message