lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From adrena.keat...@nl.abnamro.com
Subject Lucene/Tomcat Memory Leak Issue
Date Thu, 17 Aug 2006 12:19:58 GMT
Thanks Kieran,

Yep iv seen this patch before but our organisation has grave concerns
putting it into a production environment as its not verified/support by
Lucene as an official patch nor have they verified that there is an actual
bug.

Do you know if you or anyone has actuall installed in prod env and where
there any problems?

Kind Regards
Adrena Keating



                                                                                         
                                             
                      kieran                                                             
                                             
                      <kieran@kitsite.         To:      java-user@lucene.apache.org   
                                                
                      com>                     cc:                                    
                                                
                                               Subject: Re: Lucene/Tomcat Memory Leak Issue
                                           
                      08/17/2006 12:00                                                   
                                             
                      PM                                                                 
                                             
                      Please respond                                                     
                                             
                      to java-user                                                       
                                             
                                                                                         
                                             




Hello Adrena,

You might want to consult the following thread:

  http://issues.apache.org/jira/browse/LUCENE-436

The issue /might/ be to do with your JVM's implementation of ThreadLocal.
The above discussion contains a fix/workaround for this issue.

Kieran



adrena.keating@nl.abnamro.com wrote:
> We are indexing the file server via Lucene.  We have a 15 MB index file
and
> have set up a once a day re-index and then switch to a continuous index
> update.
> So everytime a content item is published it is immediately indexed and
> deployed.  Problem persists with both scenario's.
>
> We are using jconsole as a profiler but this does not give us information
> if tomcat objects are released from their references and so are not
garbage
> collected.
> Certainly its looks like the long-term heap memory is the issue and is
not
> being garbage collected effectively.
>
> We dont even know what the likely culprit is.  i.e memory leak,
heap/pergen
> memory settings, lucene bug, bug in the way the code is interacting with
> lucene.
>
> We have been looking at this problem for months now..
>
> We dont index xml files with xalan.
>
> Kind Regards
> Adrena Keating
>
>
>
>
>

>                       Ronnie Kolehmainen

>                       <ronnie.kolehmaine         To:
java-user@lucene.apache.org

>                       n@ub.uu.se>                cc:

>                                                  Subject: Re:
Lucene/Tomcat Memory Leak Issue
>                       08/11/2006 06:23

>                       PM

>                       Please respond to

>                       java-user

>

>
>
>
>
> How do you index your documents? Are you releasing old resources? Can you
> use a
> profiler to see referenced objects?
>
> I've experienced the same problem when indexing xml files which were
parsed
> with
> xalan, and the memory leak in that case was in xalan. Switching to saxon
> solved
> the problems for us.
>
> /Ronnie
>
> Citerar adrena.keating@nl.abnamro.com:
>
>
>> Hello, can anyone help?
>>
>>
>> We're experiencing the following issue an Widows Intranet website:
>>
>>
>> Following a Tomcat restart, our application has Lucene creating a single
>> new index in a RAMDirectory, followed by continuous creation of
>>
> additional
>
>> index entries as new content is published.
>>
>>
>> During the initial index creation process, Heap Memory utilization jumps
>>
> to
>
>> over 300 MB and never drops back down, even though the new index itself
>>
> is
>
>> ~15 MB in size. Then, as additional index entries are added, the Heap
>> Memory utilization continues to increase and garbage collection reduces
>>
> in
>
>> size, until we max out the available Heap Memory and Tomcat crashes.
>>
>>
>> Relevant configuration items:
>>         Windows 2000 5.0
>>         4 Processors, 2GB RAM
>>         Tomcat 5.0.27
>>         JDK 1.5.0_06
>>         Lucene 1.9.1
>>         VM arguments:
>>                 -Djava.io.tmpdir=D:\Appldata\Lucene\lockDir
>>                 -Dorg.apache.lucene.commitLockTimeout=60000
>>                 -Dorg.apache.lucene.writeLockTimeout=60000
>>                 -XX:PermSize=256m
>>                 -XX:MaxPermSize=512m
>>                 -Xrs
>>                 -Xms256m
>>                 -Xmx1024m
>>                 -Xss512k
>>         lucene.searcher.keep.open=true
>>         lucene.searcher.retries=6
>>
>>
>> Has anyone experienced similar conditions? Any suggestions?
>>
>> Kind Regards
>> Adrena Keating
>>
>>
>>
>>
>>
>
---------------------------------------------------------------------------
>
>> This message (including any attachments) is confidential and may be
>> privileged. If you have received it by mistake please notify the sender
>>
> by
>
>> return e-mail and delete this message from your system. Any unauthorised
>> use or dissemination of this message in whole or in part is strictly
>> prohibited. Please note that e-mails are susceptible to change. ABN AMRO
>> Bank N.V, which has its seat at Amsterdam, the Netherlands, and is
>> registered in the Commercial Register under number 33002587, including
>>
> its
>
>> group companies, shall not be liable for the improper or incomplete
>> transmission of the information contained in this communication nor for
>>
> any
>
>> delay in its receipt or damage to your system. ABN AMRO Bank N.V. (or
its
>> group companies) does not guarantee that the integrity of this
>> communication has been maintained nor that this communication is free of
>> viruses, interceptions or interference.
>>
>>
>
---------------------------------------------------------------------------
>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>
>
>
---------------------------------------------------------------------------
> This message (including any attachments) is confidential and may be
> privileged. If you have received it by mistake please notify the sender
by
> return e-mail and delete this message from your system. Any unauthorised
> use or dissemination of this message in whole or in part is strictly
> prohibited. Please note that e-mails are susceptible to change. ABN AMRO
> Bank N.V, which has its seat at Amsterdam, the Netherlands, and is
> registered in the Commercial Register under number 33002587, including
its
> group companies, shall not be liable for the improper or incomplete
> transmission of the information contained in this communication nor for
any
> delay in its receipt or damage to your system. ABN AMRO Bank N.V. (or its
> group companies) does not guarantee that the integrity of this
> communication has been maintained nor that this communication is free of
> viruses, interceptions or interference.
>
---------------------------------------------------------------------------
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
>



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org




---------------------------------------------------------------------------
This message (including any attachments) is confidential and may be
privileged. If you have received it by mistake please notify the sender by
return e-mail and delete this message from your system. Any unauthorised
use or dissemination of this message in whole or in part is strictly
prohibited. Please note that e-mails are susceptible to change. ABN AMRO
Bank N.V, which has its seat at Amsterdam, the Netherlands, and is
registered in the Commercial Register under number 33002587, including its
group companies, shall not be liable for the improper or incomplete
transmission of the information contained in this communication nor for any
delay in its receipt or damage to your system. ABN AMRO Bank N.V. (or its
group companies) does not guarantee that the integrity of this
communication has been maintained nor that this communication is free of
viruses, interceptions or interference.
---------------------------------------------------------------------------


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message