lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jan Philipp Seng" <>
Subject how to free memory after index ist build.
Date Tue, 02 Aug 2005 13:30:57 GMT
I am using the Lucene 1.4.3 API. After building the index over 150000
documents (~250 MB data), Lucene does not free the memory that is used
during indexing. The searcher runs as a servlet under Tomcat. Every time the
index is build new, the indexing process takes free memory, so after ten
runs the memory is completly full. I tried to call the garbage collector
explicitly, but that does not help.
I build the index to harddisc and load it into a RAMDirectory after
building. There is no reference to my indexer left after indexing and I
cannot find a reason why the garbage collector does not free the memory.
Here are some important points of my code reduced to the central
functionality. Do you have any ideas, what kind of problem this could be ?
An answer would help me a lot.

IndexTablesDaemon start the indexing process:
while (true) {
  IndexTables indexer = new IndexTables();
  indexer.indexTables(); // indexing if necessary, writing the new index to
  indexer = null;        // for freeing the memory, does not help
  FTS.initNewIndex();    // switching the old with the new index
  Runtime.getRuntime().gc();   // calling the garbage collector. Does not
  Thread.sleep(lWait);         // waiting a fix space of time

Building the index: sending queries to a mySQLDatabase, 
										buidling a Lucene-document with the mySQL-data und indexing it:
  IndexWriter writer = new IndexWriter(PATHNAME_INDEX_NEW), 
    new TTAnalyser(), true);		

  writer.mergeFactor = 250;
  writer.minMergeDocs = 250;
  Document doc = null;

  String sQuery = "SELECT columns FROM table";

  Connection conn = DriverManager.getConnection("jdbc:mysql: ...");
  Statement stmt = conn.createStatement()
  ResultSet	rs = stmt.executeQuery(sQuery);

  while ( {
    doc = getTTDocument();
    // fill doc with fields from the database query

  writer = null;              // for freeing memory. Does not help

  Runtime.getRuntime().gc();  // explicitely running the garbage collector.
Does not help
Exchanging the old an the new index for queries to the Lucene-index.
  File oldIndex = new File(sPathIndex);
  File newIndex = new File(sPathIndexNew);
  // deleting all files in the old index from harddisc
  // let the new index become the operating index
  // load the new index to RAMDirectory from harddisc
Thanks for your help,

Jan Philipp Seng, Germany, Aachen

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message