lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Rochkind <rochk...@jhu.edu>
Subject Re: Can Solr with the StatsComponent analyze 20+ million files?
Date Mon, 08 Aug 2011 21:16:58 GMT
On 8/8/2011 5:10 PM, Markus Jelsma wrote:
>> Will the StatsComponent in Solr do what we need with minimal configuration?
>> Can the StatsComponent only be used on a subset of the data? For
>> example, only look at data from certain months?
> If i remember correctly, it cannot.

Well, if you index things properly, you could an fq to only certain 
months, and then use StatsComponent on top.

But I'd agree with others that Solr is probably not the best tool for 
this job. Solr's primary area of competency is text indexing and text 
search, not mathematical calculation. If you need a whole lot of text 
indexing and a little bit of math too, you might be able to get 
StatsComponent to do what you need, although you'll probably run into 
some tricky parts becuase this isn't really Solr's focus.

But if you need a whole bunch of math and no text indexing at all -- use 
a tool that has math rather than text search as it's prime area of 
competency/focus, don't make things hard for yourself by using the wrong 
tool for the job.

(StatsComponent, incidentally, performs not-so-great on very large 
result sets, depending on what you ask it for).

Mime
View raw message