lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <>
Subject Re: potential indexing perormance improvement for compound index - cut IO - have more files though
Date Sat, 16 Dec 2006 17:35:40 GMT
Otis Gospodnetic wrote:
> I think Doron is right on the money here.  I know one "customer" who'd be happy to trade
its file descriptors for less IO -  It's exactly what Doron describes - a busy
system with a LOT of indices.  File descriptors are kept under control by carefully closing
IndexSearchers, plus I can always increase the max open-files limit.  What I can't easily
increase is the disk IO.  Sure, I could go from CFS to the multi-file format, but it would
be nice to have that third, middle ground choice.

The problem is that adding that middle ground isn't free: it will 
complicate the code and make it harder to maintain and evolve.  If you 
have good control over file handles, then non-compound format should 
work just fine, no?

I'm not yet convinced that the costs of this mid-point justify its 
benefits.  Perhaps the changes are simpler than I imagine.  Perhaps it 
can be done very simply and elegantly with little impact on the code. 
If so, then my concerns will be reduced.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message