lucenenet-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "George Aroush" <geo...@aroush.net>
Subject Memory leak (was: RE: Lucene.Net Indexing Large Databases)
Date Tue, 12 Sep 2006 01:40:04 GMT
Thanks Neil.

Per my response to Eyal, the CLR Profile did show that I am looking at the
right suspected, but it's not showing me who is holding on the references.
Maybe it's there but I haven't figured out how to use it?

-- George

-----Original Message-----
From: Michael Mitiaguin [mailto:mitiaguin@gmail.com] 
Sent: Sunday, September 10, 2006 9:54 PM
To: lucene-net-user@incubator.apache.org
Subject: Re: Lucene.Net Indexing Large Databases

George,

Microsoft provides a free CLR profiler. Can't say anything since never used.

http://www.microsoft.com/downloads/details.aspx?familyid=A362781C-3870-43BE-
8926-862B40AA0CD0&displaylang=en

Michael

On 9/11/06, Neil Carson <ncarson@everdreamcorp.com> wrote:
>
> George
>
> An excellent tool is YourKit CPU and memory profiler. Trivial to use, 
> 30 day trial, just gets the job done and nothing more/less. See 
> www.yourkit.com (http://www.yourkit.com/dotnet/download/index.jsp)
>
>     Neil
>
> ________________________________
>
> From: George Aroush [mailto:george@aroush.net]
> Sent: Sun 9/10/2006 5:50 PM
> To: lucene-net-user@incubator.apache.org;
> lucene-net-dev@incubator.apache.org
> Subject: RE: Lucene.Net Indexing Large Databases
>
>
>
> Hi Folks,
>
> Since last weekend, I have been trying to narrow down the problem to 
> this memory leak without much of a luck.
>
> Does anyone have a tool (or could recommend one, without costing me 
> $$) which hopefully show the source of the leak?
>
> Unlike C++ code, the leak here, obviously, is due to not releasing 
> references to temporary or real objects.  The trick is finding the object.
>
> This leak can be created with this simple code:
>
>         public static void  Main(System.String[] args)
>         {
>                 IndexWriter diskIndex;
>                 Directory   directory;
>                 Analyzer    analyzer;
>                 Document    doc;
>                 int         count;
>                 string      indexDirectory;
>                 System.IO.FileInfo  fi;
>
>                 indexDirectory = "C:\\Index.Bad";
>
>                 fi = new System.IO.FileInfo(indexDirectory);
>                 directory = 
> Lucene.Net.Store.FSDirectory.GetDirectory(fi,
> true);
>
>                 analyzer = new SimpleAnalyzer();
>                 diskIndex = new IndexWriter(directory, analyzer, 
> true);
>
>                 count = 0;
>                 while (count < 10000)
>                 {
>                         doc = new Document();
>                         diskIndex.AddDocument(doc);
>                         count++;
>                 }
>
>                 diskIndex.Close();
>         }
>
> This code will show a leak in 1.9, 1.9.1 and 2.0 but not 1.4.3.  I 
> also verified and it doesn't leak under the Java version of Lucene 
> (2.0 is where I tested.)
>
> Regards,
>
> -- George
>
>
> -----Original Message-----
> From: George Aroush [mailto:george@aroush.net]
> Sent: Friday, September 01, 2006 9:21 PM
> To: lucene-net-user@incubator.apache.org
> Subject: RE: Lucene.Net Indexing Large Databases
>
> Hi Chris,
>
> I am using 1.9.1 in production and I am not having this problem.  
> Sorry, I don't have enough cycles to try your code on 1.9.
>
> This problem was reported on 1.4.x and was fixed.  I am sure I carried 
> it over to 1.9.x and 2.0 -- or maybe this is a new issue.  I will 
> double check when I get the cycles.
>
> You can get 1.4.3's source code as ZIP from the download site of 
> Lucene.Net which is here:
> https://svn.apache.org/repos/asf/incubator/lucene.net/site/download/ 
> or you can SVN the source code from here:
> https://svn.apache.org/repos/asf/incubator/lucene.net/tags/
>
> Regards,
>
> -- George Aroush
>
>
> -----Original Message-----
> From: Chris David [mailto:chris@snapstream.com]
> Sent: Friday, September 01, 2006 1:46 PM
> To: lucene-net-user@incubator.apache.org
> Subject: RE: Lucene.Net Indexing Large Databases
>
> Thanks René, so its not just me with this problem.  Now where can I 
> get a hold of this wonderful 1.4 Build of Lucene.  Its not listed 
> directly on Apache's Lucene.NET Page. I am anxious to see if my code 
> actually does work.
> Thanks again for all your help, I really do appreciate it.
>
> Chris
> Snapstream Media
> -----Original Message-----
> From: René de Vries [mailto:rene.de.vries@howardshome.com]
> Sent: Friday, September 01, 2006 7:32 AM
> To: lucene-net-user@incubator.apache.org
> Subject: RE: Lucene.Net Indexing Large Databases
>
> Update: I didn't realize my earlier code example ran against 1.4.
>
> If I run this with 1.9final-005 build, I am experiencing the exact 
> same problems as Chris mentions. Memory consumtion keeps growing, I 
> had to kill it at 1.5Gb. Exact same code, but with a 1.4 version of 
> the lucene.netDLL, and it runs along at 50Mb
>
> René
>
>
>
>
>


Mime
View raw message