lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Starzer (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (LUCENE-1186) [PATCH] Clear ThreadLocal instances in close()
Date Fri, 27 Feb 2009 08:39:13 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-1186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12677281#action_12677281
] 

rviper edited comment on LUCENE-1186 at 2/27/09 12:38 AM:
------------------------------------------------------------------

i'm using quartz schedules to trigger indexing tasks

since the analyzer class is using a thread local -> should i reuse an analyzer (e.g singleton
pattern) in this case (quartz job reusing threads (=tread pool) -> thread local data never
gets freed)...)? 

Mem dump (part):

Class name Shallow Heap Retained HeapPercentage 

org.quartz.simpl.SimpleThreadPool$WorkerThread @ 0xb4e8070  QuartzScheduler_Worker-3 Thread
      120  55.914.144    21,65% 
|- java.lang.ThreadLocal$ThreadLocalMap @ 0xdf32a20      24  55.913.760  21,65% 
|  '- java.lang.ThreadLocal$ThreadLocalMap$Entry[16384] @ 0x163a61b8      65.552  55.913.736
21,65% 
java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0xe090ca8  32 9.608 0,00% 
java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0xe0f6b88     ....
java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0xe5d3df8   ....
java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0xe629d80 ....
java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0xeb53510 ....
......
...... 



      was (Author: rviper):
    i'm using quartz schedules to trigger indexing tasks

since the analyzer class is using a thread local -> should i reuse an analyzer (e.g singleton
pattern) in this case (quartz job reusing threads (=tread pool) -> thread local data never
gets freed)...)? 

Mem dump (part):

Class name                                                                               
 | Shallow Heap | Retained Heap | Percentage
--------------------------------------------------------------------------------------------------------------------------------------
org.quartz.simpl.SimpleThreadPool$WorkerThread @ 0xb4e8070  QuartzScheduler_Worker-3 Thread|
         120 |    55.914.144 |     21,65%
|- java.lang.ThreadLocal$ThreadLocalMap @ 0xdf32a20                                      
 |           24 |    55.913.760 |     21,65%
|  '- java.lang.ThreadLocal$ThreadLocalMap$Entry[16384] @ 0x163a61b8                     
 |       65.552 |    55.913.736 |     21,65%
|     |- java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0xe090ca8                          
 |           32 |         9.608 |      0,00%
|     |  |- org.apache.lucene.analysis.StopAnalyzer$SavedStreams @ 0xe090cc8             
 |           24 |         9.560 |      0,00%
|     |  |  |- org.apache.lucene.analysis.LowerCaseTokenizer @ 0xe090ce0                 
 |           32 |         8.520 |      0,00%
......
...... 


  
> [PATCH] Clear ThreadLocal instances in close()
> ----------------------------------------------
>
>                 Key: LUCENE-1186
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1186
>             Project: Lucene - Java
>          Issue Type: Improvement
>    Affects Versions: 2.3, 2.3.1, 2.4
>         Environment: any
>            Reporter: Christian Kohlschütter
>            Priority: Minor
>         Attachments: LUCENE-1186-SegmentReader.patch
>
>
> As already found out in LUCENE-436, there seems to be a garbage collection problem with
ThreadLocals at certain constellations, resulting in an OutOfMemoryError.
> The resolution there was to remove the reference to the ThreadLocal value when calling
the close() method of the affected classes (see FieldsReader and TermInfosReader).
> For Java < 5.0, this can effectively be done by calling threadLocal.set(null); for
Java >= 5.0, we would call threadLocal.remove()
> Analogously, this should be done in *any* class which creates ThreadLocal values
> Right now, two classes of the core API make use of ThreadLocals, but do not properly
remove their references to the ThreadLocal value
> 1. org.apache.lucene.index.SegmentReader
> 2. org.apache.lucene.analysis.Analyzer
> For SegmentReader, I have attached a simple patch.
> For Analyzer, there currently is no patch because Analyzer does not provide a close()
method (future to-do?)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message