lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcus Falck" <marcus.fa...@observer.se>
Subject Flushing RAMDir into FSDir
Date Wed, 28 Jun 2006 09:45:40 GMT
Hi,

 

I got a lucene based host application that retrieves content for
indexing from fetcher applications.

 

Since I get fresh content all the time I wanted to have full control
over the disc write process.

 

So I ended up using a RAMDirectory and a FSDirectory. 

 

When the content arrives to the application a representation of the
content is written to the disc and the content itself are added as a
Lucene document into the RAMDirectory. When the amount of documents in
the RAMDirectory exceeds my limit I flush the RAMDirectory into the
FSDirectory (using the same limit as minMergeDocs). 

 

This works fine as long as I use the 

fsIndexWriter.AddIndexes(new Directory[] { myRamDir } ); 

approach. However this approach seems to add the newly content to a new
large file  (as long as maxMergeDocs hasn't been exceed). 

 

For example if I got 5000 as minMergeDocs and 15000 as maxMergeDocs  and
I flush the interval 5001-10000 from the ramdir to disc I will end up
with 1 file containing docs 1 - 10000 rather than 2 files ( 1 - 5000 and
5001 - 10000 ).

 

Is there any way to get it to add the new indexes as a new file instead?

Since this severely affects my indexing in a negative way.

 

 

/

Regards 

Marcus Falck

 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message