lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <ted.dunn...@gmail.com>
Subject Re: SolrCloud - Using collections, slices and shards in the wild
Date Wed, 10 Feb 2010 18:47:35 GMT
This is analogous to the multiple data sources that we have at
Deepdyve<http://www.deepdyve.com>.
In a fully sharded and balanced environment, I have found it *much* more
efficient to put all data sources into a single collection and use a filter
to select one or the other.  The rationale is that data sources are
distributed in size according to a rough long-tail distribution.  For the
largest ones, the filters are about as efficient as a separate index because
they are such a large fraction of the index.  For the small ones, the
filtered query is so fast that other issues form the bottleneck anyway.  The
operational economies of not managing hundreds of indexes and the much
better load balancing makes the integrated solution massively better for
me.  We currently use Katta and this system works really, really well.

One big difference in our environments is that for me, the dominant query
pattern involves most data sources while for you, the dominant pattern will
likely involve a single data source.

On Tue, Feb 9, 2010 at 9:02 PM, Jon Gifford <jon.gifford@gmail.com> wrote:

> 1) Support one index per customer, and many customers (thus, many
> independent indices)
>



-- 
Ted Dunning, CTO
DeepDyve

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message