lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aristedes Maniatis <...@ish.com.au>
Subject loading zookeeper data
Date Fri, 22 Jul 2016 07:22:33 GMT
Hi everyone

I'm not new to Solr, but I'm upgrading from Solr 4 to 5 and needing to use the new Zookeeper
configuration requirement. It is adding a lot of extra complexity to our deployment and I
want to check that we are doing it right.


1. We are using Saltstack to push files to deployment servers. That makes it easy to put files
anywhere I want, run scripts, etc. If you don't know Salt, it is a lot like Puppet or other
configuration management tools. Salt is all python.

2. We use Jenkins to build and test

3. Deployment servers are all FreeBSD.


Now, in the old days, I could just push the right core configuration files to each Solr instance
(we have three cores), make sure one is the master and use cron to ensure the master updates.
The other Solr slaves all update nicely. The problem we want to escape is that this configuration
causes outages and other random issues each time the Solr master does a full reload. It shouldn't,
but it does and hopefully the new SolrCluster will be better.


Now, I can still deploy Solr and Zookeeper using Salt. All that works well and is easy. But
how I do get the configuration files from our development/test environment (built and tested
with Jenkins) into production? Obviously I want those config files in version control. And
maybe Jenkins can zip up the 8 configuration files (per core) and push them to our artifact
repository.

But then what? In the production cluster it seems I then need to

1. Grab the latest configuration bundle for each core and unpack them
2. Launch Java
3. Execute the Solr jars (from the production server since it must be the right version)
- with org.apache.solr.cloud.ZkCLI
- and some parameters pointing to the production Zookeeper cluster
- pointing also to the unpacked config files
4. Parse the output to understand if any error happened
5. Wait for Solr to pick up the new configuration and do any final production checks


Am I missing some really simple step, or is this what we must now do?

I'm thinking that gradle might help with 2&3 above since then at least it can launch the
right version of Java, download the right Solr version and execute against that. And maybe
that can run from Jenkins as a "release" step.

Is that a good approach?

Cheers
Ari




-- 
-------------------------->
Aristedes Maniatis
CEO, ish
https://www.ish.com.au
GPG fingerprint CBFB 84B4 738D 4E87 5E5C  5EFA EF6A 7D2E 3E49 102A


Mime
View raw message