lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject RE: OutOfMemoryError with Lucene 1.4 final
Date Fri, 10 Dec 2004 17:24:08 GMT
Ying,

You should follow this finally block advice below.  In addition, I
think you can just close the reader, and it will close the underlying
stream (I'm not sure about that, double-check it).

You are not running out of file handles, though.  Your JVM is running
out of memory.  You can play with:

1) -Xms and -Xmx JVM command-line parameters
2) IndexWriter's parameters: mergeFactor and minMergeDocs - check the
Javadocs for more info.  They will let you control how much memory your
indexing process uses.

Otis


--- Sildy Augustine <saugustine@interwoven.com> wrote:

> I think you should close your files in a finally clause in case of
> exceptions with file system and also print out the exception. 
> 
> You could be running out of file handles.
> 
> -----Original Message-----
> From: Jin, Ying [mailto:yingjin@indiana.edu] 
> Sent: Friday, December 10, 2004 11:15 AM
> To: lucene-user@jakarta.apache.org
> Subject: OutOfMemoryError with Lucene 1.4 final
> 
> Hi, Everyone,
> 
>  
> 
> We're trying to index ~1500 archives but get OutOfMemoryError about
> halfway through the index process. I've tried to run program under
> two
> different Redhat Linux servers: One with 256M memory and 365M swap
> space. The other one with 512M memory and 1G swap space. However,
> both
> got OutOfMemoryError at the same place (at record 898). 
> 
>  
> 
> Here is my code for indexing:
> 
> ===============================================
> 
>     Document doc = new Document();
> 
>     doc.add(Field.UnIndexed("path", f.getPath()));
> 
>     doc.add(Field.Keyword("modified",
> 
>  
> DateField.timeToString(f.lastModified())));
> 
>     doc.add(Field.UnIndexed("eprintid", id));
> 
>     doc.add(Field.Text("metadata", metadata));
> 
>  
> 
>     FileInputStream is = new FileInputStream(f);  // the text file
> 
>     BufferedReader reader = new BufferedReader(new
> InputStreamReader(is));
> 
>  
> 
>     StringBuffer stringBuffer = new StringBuffer();
> 
>     String line = "";
> 
>     try{
> 
>       while((line = reader.readLine()) != null){
> 
>         stringBuffer.append(line);
> 
>       }
> 
>       doc.add(Field.Text("contents", stringBuffer.toString()));
> 
>       // release the resources
> 
>       is.close();
> 
>       reader.close();
> 
> }catch(java.io.IOException e){}
> 
> =================================================
> 
> Is there anything wrong with my code or I need more memory?
> 
>  
> 
> Thanks for any help!
> 
> Ying
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message