lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <>
Subject Re: idea for reducing file handle use
Date Tue, 23 Sep 2003 19:20:15 GMT
Dmitry Serebrennikov wrote:
> Ok, I am working on a version that would limit the changes to the 
> Directory class, but this directory would have to make certain 
> assumptions about the names of the files (whereas right now it doesn't 
> care). It would have to differentiate the segments file, the deleted 
> documents file(s?), and the other segment files. It would also have to 
> assume that the part before the last "." in a file name is the segment 
> name. Does this sound better than the other idea?

That sounds a little ugly.

Perhaps the Directory API could be extended to better support your 
technique.  For example, one could add subdirectory notion.  One could 
create a new subdirectory for each segment, and then explicitly close it 
once the segment is complete.  On close, it could be optimized by 
appending its files into a single file and writing a table-of-contents file.

If the indexing code were changed as above, would you still need to know 
anything about the segments file or deletions files?  It seems to me 
that deletions could be handled by adding a new file into the 
subdirectory, so that a subdirectory contains both the optimized 
content, and any files added afterwards, or somesuch.

Does this seem like a workable approach?


View raw message