lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@lucene.com>
Subject Re: idea for reducing file handle use
Date Tue, 23 Sep 2003 19:13:39 GMT
Dmitry Serebrennikov wrote:
>> It would be cleaner if this could be done entirely as a Directory 
>> implementation.  I know some folks who've implemented a 
>> filesystem-within-a-file solution for this problem that they're very 
>> happy with.  It is a Directory, and requires no changes to Lucene.  
>> I'll ask them if they're willing to contribute it, so that others can 
>> use it. 
> 
> That would be great! I thought about doing this at the directory level, 
> but I wasn't sure how to handle re-writing of multi-file segments into 
> single-file segments once the IndexWriter is closed. I'd like to avoid 
> dealing with file fragmentation, so I like having multiple files during 
> IndexWriter operation. But if there is a ready-made solution that does 
> deal with fragmentation, I guess that would work just fine.

Unfortunately their solution does not deal very well with fragmentation. 
  It could be improved to do so, but it does not at present.  When 
allocating blocks to a file it could look for nearby blocks, and it 
could also be made to support an explicit defragmenting operation.

Doug


Mime
View raw message