Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of tim.potter@lucidworks.com
 designates 207.46.163.186 as permitted sender)
From: Tim Potter <tim.potter@lucidworks.com>
To: "solr-user@lucene.apache.org" <solr-user@lucene.apache.org>
Subject: RE: Replicating Between Solr Clouds
Thread-Topic: Replicating Between Solr Clouds
Thread-Index: AQHPN+UNcK2TELRns0+ocN2nlbJKCZrRyGYI
Date: Wed, 5 Mar 2014 02:51:21 +0000
Message-ID: 
 <abc333bb0b834a09998db8c9f2eff082@BLUPR06MB417.namprd06.prod.outlook.com>
References: <1393963477224-4121196.post@n3.nabble.com>
In-Reply-To: <1393963477224-4121196.post@n3.nabble.com>
Accept-Language: en-US
Content-Language: en-US
received-spf: None (: lucidworks.com does not designate permitted sender
 hosts)
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Unfortunately, there is no out-of-the-box solution for this at the moment. =
=0A=
=0A=
In the past, I solved this using a couple of different approaches, which we=
ren't all that elegant but served the purpose and were simple enough to all=
ow the ops folks to setup monitors and alerts if things didn't work.=0A=
=0A=
1) use DIH's Solr entity processor to pull data from one Solr to another, s=
ee: http://wiki.apache.org/solr/DataImportHandler#SolrEntityProcessor=0A=
=0A=
This only works if you store all fields, which in my use case was OK becaus=
e I also did lots of partial document updates, which also required me to st=
ore all fields=0A=
=0A=
2) use the replication handler's snapshot support to create snapshots on a =
regular basis and then move the files over the network=0A=
=0A=
This one works but required the use of read and write aliases and two colle=
ctions on the remote (slave) data center so that I could rebuild my write c=
ollection from the snapshots and then update the aliases to point the reads=
 at the updated collection. Work on an automated backup/restore solution is=
 planned, see https://issues.apache.org/jira/browse/SOLR-5750, but if you n=
eed something sooner, you can write a backup driver using SolrJ that uses C=
loudSolrServer to get the address of all the shard leaders, initiate the ba=
ckup command on each leader, poll the replication details handler for snaps=
hot completion on each shard, and then ship the files across the network. O=
bviously, this isn't a solution for NRT multi-homing ;-)=0A=
=0A=
Lastly, these aren't the only ways to go about this, just wanted to share s=
ome high-level details about what has worked.=0A=
=0A=
Timothy Potter=0A=
Sr. Software Engineer, LucidWorks=0A=
www.lucidworks.com=0A=
=0A=
________________________________________=0A=
From: perdurabo <robert_parker@volusion.com>=0A=
Sent: Tuesday, March 04, 2014 1:04 PM=0A=
To: solr-user@lucene.apache.org=0A=
Subject: Replicating Between Solr Clouds=0A=
=0A=
We are looking to setup a highly available failover site across a WAN for o=
ur=0A=
SolrCloud instance.  The main production instance is at colo center A and=
=0A=
consists of a 3-node ZooKeeper ensemble managing configs for a 4-node=0A=
SolrCloud running Solr 4.6.1.  We only have one collection among the 4 core=
s=0A=
and there are two shards in the collection, one master node and one replica=
=0A=
node for each shard.  Our search and indexing services address the Solr=0A=
cloud through a load balancer VIP, not a compound API call.=0A=
=0A=
Anyway, the Solr wiki explains fairly well how to replicate single node Sol=
r=0A=
collections, but I do not see an obvious way for replicating a SolrCloud's=
=0A=
indices over a WAN to another SolrCloud.  I need for a SolrCloud in another=
=0A=
data center to be able to replicate both shards of the collection in the=0A=
other data center over a WAN.  It needs to be able to replicate from a load=
=0A=
balancer VIP, not a single named server of the SolrCloud, which round robin=
s=0A=
across all four nodes/2 shards for high availability.=0A=
=0A=
I've searched high and low for a white paper or some discussion of how to d=
o=0A=
this and haven't found anything.  Any ideas?=0A=
=0A=
Thanks in advance.=0A=
=0A=
=0A=
=0A=
--=0A=
View this message in context: http://lucene.472066.n3.nabble.com/Replicatin=
g-Between-Solr-Clouds-tp4121196.html=0A=
Sent from the Solr - User mailing list archive at Nabble.com.=