lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wettin <karl.wet...@gmail.com>
Subject Re: Upper limit on number of Fields
Date Sun, 15 Feb 2009 20:55:21 GMT

15 feb 2009 kl. 16.27 skrev Joel Halbert:

> Is there any practical limit on the number of fields that can be
> maintained on an index?
>
> My index looks something like this, 1 million documents. For each  
> group
> of 1000 documents I might have 10 indexed fields. This would mean in
> total about 10000 fields. Am I going to run into any issues here?
>
> The index in Ram based.


There is a resource cost to handle identical term values in several  
fields rather than keeping them in a single field as there is one  
index per field. You might however save CPU at query time by dividing  
them up in multiple fields as a field index would be smaller and thus  
faster to navigate.  But at the same time Lucene is more or less  
logarithmic when it comes to speed so you might actually save both  
speed and resources by using the same field if you can.

I don't think there will be a problem. A million documents might  
however consume a lot of RAM. Did you measure how much slower it is on  
FS? The difference really should not be that great, especially when  
the index is that large. Also, reports (including my own) indicate  
that running the index on SSD is  as fast as running in RAM so you  
might want to look in to how much bang for the buck you get by hosting  
in RAM rather than on SSD.

Hope this helps,

        karl

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message