zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <apa...@elyograg.org>
Subject Re: Split a large ZooKeeper cluster into multiple separate clusters
Date Wed, 07 Sep 2016 23:46:08 GMT
On 9/7/2016 5:15 PM, Eric Young wrote:
> Also, adding/removing ZooKeeper nodes can be problematic to manage
> over a large cluster (partly because 3.4.6 doesn't support live config
> changes b3.5.0+ does). 

When an actual stable 3.5 release comes out, Solr can begin the process
of upgrading its ZK support to include dynamic clusters.

> Unfortunately, the final goal is to have each final SolrCloud cluster
> to have knowledge of every static collections, but the local ZooKeeper
> clusters should not know about the others in other clusters
> (effectively duplicating the collection in each cluster). So, there is
> no rearranging to do, only removing "extra" nodes after splitting the
> ZooKeeper cluster. This may sound counter productive, but the static
> collections are managed outside of Solr. In the event that I do need
> to update the content in one, I can reload the collection on per
> location basis for a less risky deployment. It's a bit scary when you
> need to reload a large static collection across 20+ Solr servers.

If I correctly understand what you're trying to do with the static
collections, Solr cannot do it directly.  If you have collection A in
cluster 1, and collection A in cluster 2, they are entirely separate and
cannot be managed as a single collection.  Queries and updates sent to
one cluster will remain in that cluster and will not ever be sent to
other clusters.

You might be able to use the CDCR (cross-data-center-replication)
feature that's new in 6.x to keep a collection in one cluster in sync
with another cluster.  I'm not familiar with the feature, but there is
documentation.

Thanks,
Shawn


Mime
View raw message