lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From petite_abeille <petite_abei...@mac.com>
Subject Homogeneous vs Heterogeneous indexes (was: FileNotFoundException)
Date Mon, 29 Apr 2002 17:54:43 GMT
First of, thanks to Jagadesh Nandasamy who directed me to the right 
direction.

It seems, that in my situation, more homogeneous indexes work better 
than fewer heterogeneous indexes:

I have a dozen class that I'm indexing. They vary from two fields to 
more than a dozen field per document (aka object). I went through 
different indexing strategy with them (per class, per date, per root 
class, ... ) to see how it goes. In any case, while trying to use my 
stuff with rc4 I consolidated all my different class indexes into one 
root class index to see if I could reduce my resources consumption. Less 
indexes, less RandomAccessFile was the rational. Well, I was wrong. In 
fact the exact opposite seems to hold true: more -homogeneous- indexes 
use overall less RandomAccessFile than less -heterogeneous- indexes...

One of those -not so obvious- thing you have to learn the hard way I 
guess... ;-)

In any case, I would like to thanks again Jagadesh for his insight. Also 
thanks to Pier Fumagalli for pointing out "LSOF". A very handy tool 
indeed.

As a final note, several people suggested to increase the number of file 
descriptors per process with something like "ulimit"... From what I 
learned today, I think it's a *bad* idea to have to change some system 
parameters just because your/my app is written in such a way that it may 
run out of some system resources. Your/my app has to fit in the system. 
Hacking "ulimit" and/or other system parameters is just a quick patch 
that will -at best- delay dealing with the real problem that's usually 
one of design.

Just my two cents.

PA.



--
To unsubscribe, e-mail:   <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>


Mime
View raw message