lucenenet-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dean Harding" <dean.hard...@dload.com.au>
Subject RE: Memory leak (was: RE: Lucene.Net Indexing Large Databases)
Date Tue, 12 Sep 2006 02:20:16 GMT
> The difference was that the implementation in Java used a WeakHashMap vs.
> the C# version used a straight Hashtable.  Java's WeakHashMap uses weak
> references; the Hashtable has strong references

.NET has a WeakReference class, though I don't think there's a built-in
"WeakHashtable" class. Shouldn't be impossible to write one...

Dean.


> -----Original Message-----
> From: Jeff Rodenburg [mailto:jeff.rodenburg@gmail.com]
> Sent: Tuesday, 12 September 2006 11:59 am
> To: lucene-net-user@incubator.apache.org
> Subject: Re: Memory leak (was: RE: Lucene.Net Indexing Large Databases)
> 
> 
> -- j
> 
> 
> On 9/11/06, George Aroush <george@aroush.net> wrote:
> >
> > Hi Eyal,
> >
> > If you run this sample program (see email thread) using Lucene.Net
> 1.4.3you
> > will not see any memory growth, but if you try with 1.9, 1.9.1 and 2.0,
> > memory will consistently grow.
> >
> > Also, if you port this program to the Java version, again, using 1.4.3
> or
> > 2.0 (I didn't try 1.9 or 1.9.1) you won't see any memory growth.
> >
> > Clearly, we have a leak somewhere in Lucene.Net 1.9, 1.9.1 and 2.0.
> There
> > was a similar problem with 1.4.3 and the fix was adding a Close() --
> > sorry,
> > I don't remember in which method, but if you are interested you can look
> > it
> > up on dotLucne when it was at SourceForge.  I should point out that the
> > Java
> > version doesn't need the Close() call.
> >
> > Regards,
> >
> > -- George
> >
> > -----Original Message-----
> > From: Eyal Post [mailto:eyalpost@epocalipse.com]
> > Sent: Monday, September 11, 2006 1:19 PM
> > To: lucene-net-user@incubator.apache.org
> > Subject: RE: Lucene.Net Indexing Large Databases
> >
> > Hi George,
> >
> > Can you explain how do you see that there's actually a leak?
> >
> > Eyal
> >
> > > -----Original Message-----
> > > From: George Aroush [mailto:george@aroush.net]
> > > Sent: Monday, September 11, 2006 3:50 AM
> > > To: lucene-net-user@incubator.apache.org;
> > > lucene-net-dev@incubator.apache.org
> > > Subject: RE: Lucene.Net Indexing Large Databases
> > >
> > > Hi Folks,
> > >
> > > Since last weekend, I have been trying to narrow down the problem to
> > > this memory leak without much of a luck.
> > >
> > > Does anyone have a tool (or could recommend one, without costing me
> > > $$) which hopefully show the source of the leak?
> > >
> > > Unlike C++ code, the leak here, obviously, is due to not releasing
> > > references to temporary or real objects.  The trick is finding the
> > > object.
> > >
> > > This leak can be created with this simple code:
> > >
> > >       public static void  Main(System.String[] args)
> > >       {
> > >               IndexWriter diskIndex;
> > >               Directory   directory;
> > >               Analyzer    analyzer;
> > >               Document    doc;
> > >               int         count;
> > >               string      indexDirectory;
> > >               System.IO.FileInfo  fi;
> > >
> > >               indexDirectory = "C:\\Index.Bad";
> > >
> > >               fi = new System.IO.FileInfo(indexDirectory);
> > >               directory =
> > > Lucene.Net.Store.FSDirectory.GetDirectory(fi,
> > > true);
> > >
> > >               analyzer = new SimpleAnalyzer();
> > >               diskIndex = new IndexWriter(directory, analyzer, true);
> > >
> > >               count = 0;
> > >               while (count < 10000)
> > >               {
> > >                       doc = new Document();
> > >                       diskIndex.AddDocument(doc);
> > >                       count++;
> > >               }
> > >
> > >               diskIndex.Close();
> > >       }
> > >
> > > This code will show a leak in 1.9, 1.9.1 and 2.0 but not 1.4.3.  I
> > > also verified and it doesn't leak under the Java version of Lucene
> > > (2.0 is where I tested.)
> > >
> > > Regards,
> > >
> > > -- George
> > >
> > >
> > > -----Original Message-----
> > > From: George Aroush [mailto:george@aroush.net]
> > > Sent: Friday, September 01, 2006 9:21 PM
> > > To: lucene-net-user@incubator.apache.org
> > > Subject: RE: Lucene.Net Indexing Large Databases
> > >
> > > Hi Chris,
> > >
> > > I am using 1.9.1 in production and I am not having this problem.
> > > Sorry, I don't have enough cycles to try your code on 1.9.
> > >
> > > This problem was reported on 1.4.x and was fixed.  I am sure I carried
> > > it over to 1.9.x and 2.0 -- or maybe this is a new issue.  I will
> > > double check when I get the cycles.
> > >
> > > You can get 1.4.3's source code as ZIP from the download site of
> > > Lucene.Net which is here:
> > > https://svn.apache.org/repos/asf/incubator/lucene.net/site/dow
> > nload/ or you can SVN the source code from here:
> > https://svn.apache.org/repos/asf/incubator/lucene.net/tags/
> >
> > Regards,
> >
> > -- George Aroush
> >
> >
> > -----Original Message-----
> > From: Chris David [mailto:chris@snapstream.com]
> > Sent: Friday, September 01, 2006 1:46 PM
> > To: lucene-net-user@incubator.apache.org
> > Subject: RE: Lucene.Net Indexing Large Databases
> >
> > Thanks René, so its not just me with this problem.  Now where can I get
> a
> > hold of this wonderful 1.4 Build of Lucene.  Its not listed directly on
> > Apache's Lucene.NET Page. I am anxious to see if my code actually does
> > work.
> > Thanks again for all your help, I really do appreciate it.
> >
> > Chris
> > Snapstream Media
> > -----Original Message-----
> > From: René de Vries [mailto:rene.de.vries@howardshome.com]
> > Sent: Friday, September 01, 2006 7:32 AM
> > To: lucene-net-user@incubator.apache.org
> > Subject: RE: Lucene.Net Indexing Large Databases
> >
> > Update: I didn't realize my earlier code example ran against 1.4.
> >
> > If I run this with 1.9final-005 build, I am experiencing the exact same
> > problems as Chris mentions. Memory consumtion keeps growing, I had to
> kill
> > it at 1.5Gb. Exact same code, but with a 1.4 version of the
> lucene.netDLL,
> > and it runs along at 50Mb
> >
> > René
> >
> >
> >
> >


Mime
View raw message