lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Replacing legacyCloud Behaviour in Solr7
Date Mon, 23 Oct 2017 23:31:10 GMT
Well, first you can explicitly set legacyCloud=true by using the
Collections API CLUSTERPROP command. I don't recommend this, mind you,
as legacyCloud will not be supported forever.

I'm not following something here though. When you say:
"The desired final state of a such a deployment is a fully configured
cluster ready to accept updates."
are there any documents already in the index or is this really a new collection?

and "adding new nodes requires explicit configuration"

Not sure what you mean here. Configuration of what?  Just spinning up
a Solr node pointing to the right ZooKeeper should be sufficient, or
I'm not understanding at all.

If not, your proposed outline seems right with one difference:
"if a node needs to be added: provision a machine, start up Solr, use
ADDREPLICA from Collections API passing shard number and coreNodeName"

coreNodeName isn't something you ordinarily need to bother with. I'm
being specific here where coreNodeName is usually something like
core_node7. I suspect you're really talking about the "node" parameter
to ADDREPLCIA, something like: 192.168.1.32:8983_solr, the entry from
live_nodes.

Now, all that said you may be better off just letting Solr add the
replica where it wants, it'll usually put a new replica on a node
without replicas so specifying the collection and shard should be
sufficient. Also, note that there are replica placement rules that can
help enforce this kind of thing.

Best,
Erick

On Mon, Oct 23, 2017 at 3:12 PM, Marko Babic <babmarko@abebooks.com> wrote:
> Hi everyone,
>
> I'm working on upgrading a set of clusters from Solr 4.10.4 to Solr 7.1.0.
>
> Our deployment tooling no longer works given that legacyCloud defaults to false (SOLR-8256)
and I'm hoping to get some advice on what to do going forward.
>
> Our setup is as follows:
>   * we run in AWS with multiple independent Solr clusters, each with its own Zookeeper
tier
>   * each cluster hosts only a single collection
>   * each machine/node in the cluster has a single core / is a replica for one shard in
the collection
>
> We bring up new clusters as needed.  This is entirely automated and basically works as
follows:
>   * we first provision and set up a fresh Zookeeper tier
>   * then, we provision a Solr bootstrapper machine that uploads collection config, specifies
numShards and starts up
>   * it's then easy provision the rest of the machines and have them automatically join
a shard in the collection by hooking them to the right Zookeeper cluster and specifying numShards
>   * if a node needs to be added to the cluster we just need to spin a machine up and
start up Solr
>
> The desired final state of a such a deployment is a fully configured cluster ready to
accept updates.
>
> Now that legacyCloud is false I'm not sure how to preserve this pretty nice, hands-off
deployment style as the bootstrapping performed by the first node provisioned doesn't create
a collection and adding new nodes requires explicit configuration.
>
> A new deployment procedure that I've worked out using the Collections API would look
like:
>   * provision Zookeeper tier
>   * provision all the Solr nodes, wait for them all to come up
>   * upload collection config + solr.xml to Zookeeper
>   * create collection using Collections API
>   * if a node needs to be added: provision a machine, start up Solr, use ADDREPLICA from
Collections API passing shard number and coreNodeName
>
> This isn’t a giant deal to build but it adds complexity that I'm not excited about
as deployment tooling needs to have some understanding of what the global state of the cluster
is before being able to create a collection or when adding/replacing nodes.
>
> The questions I was hoping someone would have some time to help me with are:
>
> * Does the new deployment procedure I've suggested seem reasonable?  Would we be doing
anything wrong/fighting best practices?
>   * Is there a way to keep cluster provisioning automated without having to build additional
orchestration logic into our deployment tooling (using autoscaling, or triggers, or something
I don’t know about)?
>
> Apologies for the wall of text and thanks. :)
>
> Marko
>

Mime
View raw message