lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Toke Eskildsen ...@statsbiblioteket.dk>
Subject RE: How to make SolrCloud more elastic
Date Thu, 12 Feb 2015 19:34:31 GMT
Matt Kuiper [matt.kuiper@issinc.com] wrote:
> Thanks for your reply.  Yes, I believe I will be working with a write
> once archive.  However, my understanding is that all shards are
> defined up front, with the option to split later.

Our situation might be a bit special as a few minutes downtime - preferably at off-peak hours
- now and then is acceptable.

We basically maintain a SolrCloud with static shards and use a completely separate builder
to generate new shards, one at a time. When the builder has finished a shard, we add it to
the cloud the hard way (re-configuration and restarting, hence the downtime). There's a description
at https://sbdevel.wordpress.com/net-archive-search/

To avoid too much ZooKeeper hassle, we have a bunch of empty shards, ready to be switched
with newly build ones. We have contemplated making the shard under construction being part
of the Solrcloud, but have yet to experiment with that setup.

Static shards, optimized down to a single segment and using DocValues for faceting is a very
potent mix: A Solr serving a non-static index needs more memory as it must be capable of handling
having more than one version of the index open at a time, plus the indexing itself. Faceting
on many unique values is more efficient with single-segment as there is no need for an internal
structure mapping the terms between the segments.

- Toke Eskildsen

Mime
View raw message