lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Siraj Haider <si...@jobdiva.com>
Subject Re: Garbage Collection performance on 2.9.2
Date Fri, 26 Mar 2010 14:37:43 GMT
oops,  forgot the attachments... its here now...

On 3/26/2010 10:33 AM, Siraj Haider wrote:
> Hi Mike,
> I am attaching the dump that I created by putting
> -XX:+PrintClassHistogram in catalina options and by issuing a kill -3
> command.  The machine was not in a bad state (i.e. it was not doing
> frequent Full GC cycles yet) but the kill -3 forced it to do a Full GC.
> If you want I will wait for the machine doing frequent gc cycle and will
> send you another another dump.
>
> thanks
> -siraj
>
> On 3/26/2010 9:29 AM, Michael McCandless wrote:
>    
>> I think there are various ways....
>>
>> jmap would be great.
>>
>> Hmm looks like this may be relevant:
>>
>>       http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6797870
>>
>> I also use YourKit and watch the allocations...
>>
>> Mike
>>
>> On Thu, Mar 25, 2010 at 5:26 PM, Siraj Haider<siraj@jobdiva.com>   wrote:
>>
>>      
>>> how should I get that memory dump? using jmap?
>>>
>>> -siraj
>>>
>>> On 3/25/2010 4:32 PM, Michael McCandless wrote:
>>>
>>>        
>>>> Are you using IndexReader.reopen to open those new searchers?
>>>>
>>>> Can you capture a memory dump when GC kicks in?  I'd be curious to see
>>>> where all the "new" garbage is coming from... I don't think 2.3.2 ->
>>>> 2.9.2 should be generating more garbage.
>>>>
>>>> Mike
>>>>
>>>> On Thu, Mar 25, 2010 at 3:55 PM, Siraj Haider<siraj@jobdiva.com>  
  wrote:
>>>>
>>>>
>>>>          
>>>>> Indexing happens in a different thread on intervals.  I open a new
>>>>> IndexWriter for each indexing session.  After indexing session if there
>>>>> is a
>>>>> modification in the index I close the searcher and open a new one.  I
>>>>> have
>>>>> two searchers that I flip flop when opening an index.
>>>>>
>>>>> On 3/25/2010 3:26 PM, Michael McCandless wrote:
>>>>>
>>>>>
>>>>>            
>>>>>> How do you reopen your searchers after indexing?
>>>>>>
>>>>>> Do you keep a single IW open for all time?
>>>>>>
>>>>>> Mike
>>>>>>
>>>>>> On Thu, Mar 25, 2010 at 3:11 PM, Siraj Haider<siraj@jobdiva.com>
>>>>>>    wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>              
>>>>>>> Indexing happen with frequent intervals on our indexes, but I
think
>>>>>>> searching is the cause of the issue, because as soon as the indexes
are
>>>>>>> hit
>>>>>>> with a lot of searches, the gc cycles become more frequent.
>>>>>>>
>>>>>>> -siraj
>>>>>>>
>>>>>>> On 3/24/2010 5:19 PM, Michael McCandless wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>                
>>>>>>>> Is this during indexing or searching?
>>>>>>>>
>>>>>>>> Mike
>>>>>>>>
>>>>>>>> On Wed, Mar 24, 2010 at 3:45 PM, Grant Ingersoll<gsingers@apache.org>
>>>>>>>>    wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                  
>>>>>>>>> On Mar 24, 2010, at 2:13 PM, Siraj Haider wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                    
>>>>>>>>>> We upgraded to 2.9.2 from 2.3.2 and the garbage collection
>>>>>>>>>> performance
>>>>>>>>>> deteriorated drastically.  The system is going to
Full GC cycles
>>>>>>>>>> with
>>>>>>>>>> long
>>>>>>>>>> pauses very frequently.  Did something got changed
that we need to
>>>>>>>>>> account
>>>>>>>>>> for?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                      
>>>>>>>>> Yes, quite a bit has changed under the hood between those
two
>>>>>>>>> versions.
>>>>>>>>>    CHANGES.txt is probably the best place to start. 
Otherwise, more
>>>>>>>>> info
>>>>>>>>> is
>>>>>>>>> needed to figure out what is going on.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                    
>>>>>>>>>> thanks in advance
>>>>>>>>>> -siraj
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                      
>>>>>>>>> --------------------------
>>>>>>>>> Grant Ingersoll
>>>>>>>>> http://www.lucidimagination.com/
>>>>>>>>>
>>>>>>>>> Search the Lucene ecosystem using Solr/Lucene:
>>>>>>>>> http://www.lucidimagination.com/search
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                    
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                  
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>                
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>              
>>>>>
>>>>>            
>>>
>>>        
>
>    



Mime
View raw message