lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jack Krupansky" <j...@basetechnology.com>
Subject Re: Is it OK to have very big number of fields in solr/lucene ?
Date Fri, 08 Aug 2014 22:37:56 GMT
Solr scales based on number of documents, not fields or collections. Dozens 
of fields or collections is perfectly fine. Hundreds of fields or 
collections CAN work, but you have to be extra diligent and use more 
powerful hardware. Millions and even billions of DOCUMENTS is fine - that's 
the primary way that Solr scales (that and shards.)

Dynamic fields are fine too, but dozens or hundreds for a single document 
are recommended limits. Different documents can have different dynamic 
fields, so the total field count could be thousands, although, again, you 
may have to be extra diligent and use more powerful hardware.

Architect your application and model your data around the strengths of Solr 
(and Lucene.) And also look at your queries first, to make sure they will 
make sense.

-- Jack Krupansky

-----Original Message----- 
From: Lisheng Zhang
Sent: Friday, August 8, 2014 5:25 PM
To: solr-user@lucene.apache.org
Subject: Is it OK to have very big number of fields in solr/lucene ?

In our application there are many complicated filter conditions, very often
those conditions are special to each user (like whether or not a doc is
important or already read by a user ..), two possible solutions to
implement those filters in lucene:

1/ create many fields
2/ create many collections (for each user, for example)

Here the number could be as big as 10G. I would prefer the 1st solution
(many fields), but if lucene cache all existing field info, memory could be
a problem ?

Thanks very much for helps, Lisheng 


Mime
View raw message