lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kieran <kie...@kitsite.com>
Subject Re: Lucene/Tomcat Memory Leak Issue
Date Thu, 17 Aug 2006 10:00:09 GMT
Hello Adrena,

You might want to consult the following thread:

  http://issues.apache.org/jira/browse/LUCENE-436

The issue /might/ be to do with your JVM's implementation of ThreadLocal.
The above discussion contains a fix/workaround for this issue.

Kieran



adrena.keating@nl.abnamro.com wrote:
> We are indexing the file server via Lucene.  We have a 15 MB index file and
> have set up a once a day re-index and then switch to a continuous index
> update.
> So everytime a content item is published it is immediately indexed and
> deployed.  Problem persists with both scenario's.
>
> We are using jconsole as a profiler but this does not give us information
> if tomcat objects are released from their references and so are not garbage
> collected.
> Certainly its looks like the long-term heap memory is the issue and is not
> being garbage collected effectively.
>
> We dont even know what the likely culprit is.  i.e memory leak, heap/pergen
> memory settings, lucene bug, bug in the way the code is interacting with
> lucene.
>
> We have been looking at this problem for months now..
>
> We dont index xml files with xalan.
>
> Kind Regards
> Adrena Keating
>
>
>
>
>                                                                                     
                                                    
>                       Ronnie Kolehmainen                                            
                                                    
>                       <ronnie.kolehmaine         To:      java-user@lucene.apache.org
                                                   
>                       n@ub.uu.se>                cc:                             
                                                       
>                                                  Subject: Re: Lucene/Tomcat Memory Leak
Issue                                            
>                       08/11/2006 06:23                                              
                                                    
>                       PM                                                            
                                                    
>                       Please respond to                                             
                                                    
>                       java-user                                                     
                                                    
>                                                                                     
                                                    
>
>
>
>
> How do you index your documents? Are you releasing old resources? Can you
> use a
> profiler to see referenced objects?
>
> I've experienced the same problem when indexing xml files which were parsed
> with
> xalan, and the memory leak in that case was in xalan. Switching to saxon
> solved
> the problems for us.
>
> /Ronnie
>
> Citerar adrena.keating@nl.abnamro.com:
>
>   
>> Hello, can anyone help?
>>
>>
>> We're experiencing the following issue an Widows Intranet website:
>>
>>
>> Following a Tomcat restart, our application has Lucene creating a single
>> new index in a RAMDirectory, followed by continuous creation of
>>     
> additional
>   
>> index entries as new content is published.
>>
>>
>> During the initial index creation process, Heap Memory utilization jumps
>>     
> to
>   
>> over 300 MB and never drops back down, even though the new index itself
>>     
> is
>   
>> ~15 MB in size. Then, as additional index entries are added, the Heap
>> Memory utilization continues to increase and garbage collection reduces
>>     
> in
>   
>> size, until we max out the available Heap Memory and Tomcat crashes.
>>
>>
>> Relevant configuration items:
>>         Windows 2000 5.0
>>         4 Processors, 2GB RAM
>>         Tomcat 5.0.27
>>         JDK 1.5.0_06
>>         Lucene 1.9.1
>>         VM arguments:
>>                 -Djava.io.tmpdir=D:\Appldata\Lucene\lockDir
>>                 -Dorg.apache.lucene.commitLockTimeout=60000
>>                 -Dorg.apache.lucene.writeLockTimeout=60000
>>                 -XX:PermSize=256m
>>                 -XX:MaxPermSize=512m
>>                 -Xrs
>>                 -Xms256m
>>                 -Xmx1024m
>>                 -Xss512k
>>         lucene.searcher.keep.open=true
>>         lucene.searcher.retries=6
>>
>>
>> Has anyone experienced similar conditions? Any suggestions?
>>
>> Kind Regards
>> Adrena Keating
>>
>>
>>
>>
>>     
> ---------------------------------------------------------------------------
>   
>> This message (including any attachments) is confidential and may be
>> privileged. If you have received it by mistake please notify the sender
>>     
> by
>   
>> return e-mail and delete this message from your system. Any unauthorised
>> use or dissemination of this message in whole or in part is strictly
>> prohibited. Please note that e-mails are susceptible to change. ABN AMRO
>> Bank N.V, which has its seat at Amsterdam, the Netherlands, and is
>> registered in the Commercial Register under number 33002587, including
>>     
> its
>   
>> group companies, shall not be liable for the improper or incomplete
>> transmission of the information contained in this communication nor for
>>     
> any
>   
>> delay in its receipt or damage to your system. ABN AMRO Bank N.V. (or its
>> group companies) does not guarantee that the integrity of this
>> communication has been maintained nor that this communication is free of
>> viruses, interceptions or interference.
>>
>>     
> ---------------------------------------------------------------------------
>   
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>>     
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>
>
> ---------------------------------------------------------------------------
> This message (including any attachments) is confidential and may be
> privileged. If you have received it by mistake please notify the sender by
> return e-mail and delete this message from your system. Any unauthorised
> use or dissemination of this message in whole or in part is strictly
> prohibited. Please note that e-mails are susceptible to change. ABN AMRO
> Bank N.V, which has its seat at Amsterdam, the Netherlands, and is
> registered in the Commercial Register under number 33002587, including its
> group companies, shall not be liable for the improper or incomplete
> transmission of the information contained in this communication nor for any
> delay in its receipt or damage to your system. ABN AMRO Bank N.V. (or its
> group companies) does not guarantee that the integrity of this
> communication has been maintained nor that this communication is free of
> viruses, interceptions or interference.
> ---------------------------------------------------------------------------
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>   



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message