lucene-lucene-net-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Jordan (JIRA)" <j...@apache.org>
Subject [jira] Updated: (LUCENENET-358) CloseableThreadLocal memory leak in LocalDataStoreSlot (with workaround)
Date Fri, 07 May 2010 22:38:48 GMT

     [ https://issues.apache.org/jira/browse/LUCENENET-358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Robert Jordan updated LUCENENET-358:
------------------------------------

    Attachment: CloseableThreadLocal.diff

Here a patch that uses [ThreadStatic] instead of LocalDataStoreSlot.

I don't know if it's faster, so please test it. It does pass the unit tests.

Since MS.NET does not have Java's retention issues, I've removed WeakReferences.

Don't be worried about the missing locks. They are not needed because [ThreadStatic] ensures
that "slots" is null in any new thread.


> CloseableThreadLocal memory leak  in LocalDataStoreSlot (with workaround)
> -------------------------------------------------------------------------
>
>                 Key: LUCENENET-358
>                 URL: https://issues.apache.org/jira/browse/LUCENENET-358
>             Project: Lucene.Net
>          Issue Type: Bug
>         Environment: Microsoft WIndows Server 2008 Enterprise x64. SP2.
> .NET Framework 4.0
>            Reporter: Rezgar Cadro
>            Priority: Critical
>         Attachments: CloseableThreadLocal MemoryLeak.patch, CloseableThreadLocal.diff
>
>
> Recently we have been suffering from a severe memory leak when executing intense open/close
operations on IndexSearcher and IndexModifier. 
> Memory profiling showed that memory is being held by LocalDataStore[] objects.
> After some digging, the root of the problem has been found in CloseableThreadLocal class:
> private System.LocalDataStoreSlot t = System.Threading.Thread.AllocateDataSlot();
> What we see is that every instantiated object of CloseableThreadLocal causes new data
slot allocation performed for every thread. 
> Thread.AllocateDataSlot() does not simply allocate a new slot, replacing an old one,
but enlarging an existing buffer in-thread, appending data to the end of internal LocalDataStore[]
collection, which  causes a severe memory leak .
> As long as "t" variable is instantiated on every object creation, and (in current class
implementation) every object is used by a single thread, replacing "private System.LocalDataStoreSlot
t = System.Threading.Thread.AllocateDataSlot();" with simple "private object dataSlot;" and
removing "hardRefs" Dictionary solves the problem and prevents memory leak. 
> We have tried to implement the expected behavior by using [ThreadStatic] attribute instead
of LocalDataStoreSlot, but the attempt failed because of unexpected exceptions being thrown.
> Patch can be found at Lucene.Net repository under 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message