lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Siraj Haider <si...@jobdiva.com>
Subject Re: Garbage Collection performance on 2.9.2
Date Fri, 26 Mar 2010 14:33:21 GMT
Hi Mike,
I am attaching the dump that I created by putting 
-XX:+PrintClassHistogram in catalina options and by issuing a kill -3 
command.  The machine was not in a bad state (i.e. it was not doing 
frequent Full GC cycles yet) but the kill -3 forced it to do a Full GC.  
If you want I will wait for the machine doing frequent gc cycle and will 
send you another another dump.

thanks
-siraj

On 3/26/2010 9:29 AM, Michael McCandless wrote:
> I think there are various ways....
>
> jmap would be great.
>
> Hmm looks like this may be relevant:
>
>      http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6797870
>
> I also use YourKit and watch the allocations...
>
> Mike
>
> On Thu, Mar 25, 2010 at 5:26 PM, Siraj Haider<siraj@jobdiva.com>  wrote:
>    
>> how should I get that memory dump? using jmap?
>>
>> -siraj
>>
>> On 3/25/2010 4:32 PM, Michael McCandless wrote:
>>      
>>> Are you using IndexReader.reopen to open those new searchers?
>>>
>>> Can you capture a memory dump when GC kicks in?  I'd be curious to see
>>> where all the "new" garbage is coming from... I don't think 2.3.2 ->
>>> 2.9.2 should be generating more garbage.
>>>
>>> Mike
>>>
>>> On Thu, Mar 25, 2010 at 3:55 PM, Siraj Haider<siraj@jobdiva.com>    wrote:
>>>
>>>        
>>>> Indexing happens in a different thread on intervals.  I open a new
>>>> IndexWriter for each indexing session.  After indexing session if there
>>>> is a
>>>> modification in the index I close the searcher and open a new one.  I
>>>> have
>>>> two searchers that I flip flop when opening an index.
>>>>
>>>> On 3/25/2010 3:26 PM, Michael McCandless wrote:
>>>>
>>>>          
>>>>> How do you reopen your searchers after indexing?
>>>>>
>>>>> Do you keep a single IW open for all time?
>>>>>
>>>>> Mike
>>>>>
>>>>> On Thu, Mar 25, 2010 at 3:11 PM, Siraj Haider<siraj@jobdiva.com>
>>>>>   wrote:
>>>>>
>>>>>
>>>>>            
>>>>>> Indexing happen with frequent intervals on our indexes, but I think
>>>>>> searching is the cause of the issue, because as soon as the indexes
are
>>>>>> hit
>>>>>> with a lot of searches, the gc cycles become more frequent.
>>>>>>
>>>>>> -siraj
>>>>>>
>>>>>> On 3/24/2010 5:19 PM, Michael McCandless wrote:
>>>>>>
>>>>>>
>>>>>>              
>>>>>>> Is this during indexing or searching?
>>>>>>>
>>>>>>> Mike
>>>>>>>
>>>>>>> On Wed, Mar 24, 2010 at 3:45 PM, Grant Ingersoll<gsingers@apache.org>
>>>>>>>   wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>                
>>>>>>>> On Mar 24, 2010, at 2:13 PM, Siraj Haider wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                  
>>>>>>>>> We upgraded to 2.9.2 from 2.3.2 and the garbage collection
>>>>>>>>> performance
>>>>>>>>> deteriorated drastically.  The system is going to Full
GC cycles
>>>>>>>>> with
>>>>>>>>> long
>>>>>>>>> pauses very frequently.  Did something got changed that
we need to
>>>>>>>>> account
>>>>>>>>> for?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                    
>>>>>>>> Yes, quite a bit has changed under the hood between those
two
>>>>>>>> versions.
>>>>>>>>   CHANGES.txt is probably the best place to start.  Otherwise,
more
>>>>>>>> info
>>>>>>>> is
>>>>>>>> needed to figure out what is going on.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                  
>>>>>>>>> thanks in advance
>>>>>>>>> -siraj
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                    
>>>>>>>> --------------------------
>>>>>>>> Grant Ingersoll
>>>>>>>> http://www.lucidimagination.com/
>>>>>>>>
>>>>>>>> Search the Lucene ecosystem using Solr/Lucene:
>>>>>>>> http://www.lucidimagination.com/search
>>>>>>>>
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                  
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>                
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>              
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>
>>>>>
>>>>>
>>>>>            
>>>>
>>>>          
>>
>>      



Mime
View raw message