Mailing-List: contact lucene-net-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: lucene-net-user@incubator.apache.org
Received-SPF: error (idunn.apache.osuosl.org: domain aroush.net from
 216.222.193.2 cause and error)
From: "George Aroush" <george@aroush.net>
To: <lucene-net-user@incubator.apache.org>
Subject: Memory leak (was: RE: Lucene.Net Indexing Large Databases)
Date: Mon, 11 Sep 2006 21:24:52 -0400
Message-ID: <018201c6d60a$3fed6f60$ee6ffea9@aroushlt>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
In-Reply-To: <003a01c6d5c6$6874e510$8700a8c0@srv1>
Thread-Index: 
 AcbLw/x09jKXG63cSiC7sumoZyL4tQAZ2WrwABFxkwAAF0c4sAA87lYQAArz+iAAD9Qm0AAAO9xwAcJBokAAI9G5MAAQfU2g

Hi Eyal,

If you run this sample program (see email thread) using Lucene.Net 1.4.3 =
you
will not see any memory growth, but if you try with 1.9, 1.9.1 and 2.0,
memory will consistently grow.

Also, if you port this program to the Java version, again, using 1.4.3 =
or
2.0 (I didn't try 1.9 or 1.9.1) you won't see any memory growth.

Clearly, we have a leak somewhere in Lucene.Net 1.9, 1.9.1 and 2.0.  =
There
was a similar problem with 1.4.3 and the fix was adding a Close() -- =
sorry,
I don't remember in which method, but if you are interested you can look =
it
up on dotLucne when it was at SourceForge.  I should point out that the =
Java
version doesn't need the Close() call.

Regards,

-- George

-----Original Message-----
From: Eyal Post [mailto:eyalpost@epocalipse.com]=20
Sent: Monday, September 11, 2006 1:19 PM
To: lucene-net-user@incubator.apache.org
Subject: RE: Lucene.Net Indexing Large Databases

Hi George,

Can you explain how do you see that there's actually a leak?

Eyal

> -----Original Message-----
> From: George Aroush [mailto:george@aroush.net]
> Sent: Monday, September 11, 2006 3:50 AM
> To: lucene-net-user@incubator.apache.org;
> lucene-net-dev@incubator.apache.org
> Subject: RE: Lucene.Net Indexing Large Databases
>=20
> Hi Folks,
>=20
> Since last weekend, I have been trying to narrow down the problem to=20
> this memory leak without much of a luck.
>=20
> Does anyone have a tool (or could recommend one, without costing me=20
> $$) which hopefully show the source of the leak?
>=20
> Unlike C++ code, the leak here, obviously, is due to not releasing=20
> references to temporary or real objects.  The trick is finding the=20
> object.
>=20
> This leak can be created with this simple code:
>=20
> 	public static void  Main(System.String[] args)
> 	{
> 		IndexWriter diskIndex;
> 		Directory   directory;
> 		Analyzer    analyzer;
> 		Document    doc;
> 		int         count;
> 		string      indexDirectory;
> 		System.IO.FileInfo  fi;
>=20
> 		indexDirectory =3D "C:\\Index.Bad";
>=20
> 		fi =3D new System.IO.FileInfo(indexDirectory);
> 		directory =3D
> Lucene.Net.Store.FSDirectory.GetDirectory(fi,
> true);
> 					=09
> 		analyzer =3D new SimpleAnalyzer();
> 		diskIndex =3D new IndexWriter(directory, analyzer, true);
> 	=09
> 		count =3D 0;
> 		while (count < 10000)
> 		{	=09
> 			doc =3D new Document();
> 			diskIndex.AddDocument(doc);
> 			count++;
> 		}
>=20
> 		diskIndex.Close();
> 	}
>=20
> This code will show a leak in 1.9, 1.9.1 and 2.0 but not 1.4.3.  I=20
> also verified and it doesn't leak under the Java version of Lucene=20
> (2.0 is where I tested.)
>=20
> Regards,
>=20
> -- George
>=20
>=20
> -----Original Message-----
> From: George Aroush [mailto:george@aroush.net]
> Sent: Friday, September 01, 2006 9:21 PM
> To: lucene-net-user@incubator.apache.org
> Subject: RE: Lucene.Net Indexing Large Databases
>=20
> Hi Chris,
>=20
> I am using 1.9.1 in production and I am not having this problem. =20
> Sorry, I don't have enough cycles to try your code on 1.9.
>=20
> This problem was reported on 1.4.x and was fixed.  I am sure I carried =

> it over to 1.9.x and 2.0 -- or maybe this is a new issue.  I will=20
> double check when I get the cycles.
>=20
> You can get 1.4.3's source code as ZIP from the download site of=20
> Lucene.Net which is here:
> https://svn.apache.org/repos/asf/incubator/lucene.net/site/dow
nload/ or you can SVN the source code from here:
https://svn.apache.org/repos/asf/incubator/lucene.net/tags/

Regards,

-- George Aroush


-----Original Message-----
From: Chris David [mailto:chris@snapstream.com]
Sent: Friday, September 01, 2006 1:46 PM
To: lucene-net-user@incubator.apache.org
Subject: RE: Lucene.Net Indexing Large Databases

Thanks Ren=E9, so its not just me with this problem.  Now where can I =
get a
hold of this wonderful 1.4 Build of Lucene.  Its not listed directly on
Apache's Lucene.NET Page. I am anxious to see if my code actually does =
work.
Thanks again for all your help, I really do appreciate it.

Chris
Snapstream Media
-----Original Message-----
From: Ren=E9 de Vries [mailto:rene.de.vries@howardshome.com]
Sent: Friday, September 01, 2006 7:32 AM
To: lucene-net-user@incubator.apache.org
Subject: RE: Lucene.Net Indexing Large Databases

Update: I didn't realize my earlier code example ran against 1.4.=20

If I run this with 1.9final-005 build, I am experiencing the exact same
problems as Chris mentions. Memory consumtion keeps growing, I had to =
kill
it at 1.5Gb. Exact same code, but with a 1.4 version of the lucene.net =
DLL,
and it runs along at 50Mb

Ren=E9