lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcus Falck" <>
Subject Flushing RAMDir into FSDir
Date Wed, 28 Jun 2006 09:45:40 GMT


I got a lucene based host application that retrieves content for
indexing from fetcher applications.


Since I get fresh content all the time I wanted to have full control
over the disc write process.


So I ended up using a RAMDirectory and a FSDirectory. 


When the content arrives to the application a representation of the
content is written to the disc and the content itself are added as a
Lucene document into the RAMDirectory. When the amount of documents in
the RAMDirectory exceeds my limit I flush the RAMDirectory into the
FSDirectory (using the same limit as minMergeDocs). 


This works fine as long as I use the 

fsIndexWriter.AddIndexes(new Directory[] { myRamDir } ); 

approach. However this approach seems to add the newly content to a new
large file  (as long as maxMergeDocs hasn't been exceed). 


For example if I got 5000 as minMergeDocs and 15000 as maxMergeDocs  and
I flush the interval 5001-10000 from the ramdir to disc I will end up
with 1 file containing docs 1 - 10000 rather than 2 files ( 1 - 5000 and
5001 - 10000 ).


Is there any way to get it to add the new indexes as a new file instead?

Since this severely affects my indexing in a negative way.





Marcus Falck


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message