lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Rochkind <rochk...@jhu.edu>
Subject RE: True master-master fail-over without data gaps
Date Wed, 09 Mar 2011 06:06:18 GMT
I'd honestly think about buffer the incoming documents in some store that's actually made for
fail-over persistence reliability, maybe CouchDB or something. And then that's taking care
of not losing anything, and the problem becomes how we make sure that our solr master indexes
are kept in sync with the actual persistent store; which I'm still not sure about, but I'm
thinking it's a simpler problem. The right tool for the right job, that kind of failover persistence
is not solr's specialty. 
________________________________________
From: Otis Gospodnetic [otis_gospodnetic@yahoo.com]
Sent: Tuesday, March 08, 2011 11:45 PM
To: solr-user@lucene.apache.org
Subject: True master-master fail-over without data gaps

Hello,

What are some common or good ways to handle indexing (master) fail-over?
Imagine you have a continuous stream of incoming documents that you have to
index without losing any of them (or with losing as few of them as possible).
How do you set up you masters?
In other words, you can't just have 2 masters where the secondary is the
Repeater (or Slave) of the primary master and replicates the index periodically:
you need to have 2 masters that are in sync at all times!
How do you achieve that?

* Do you just put N masters behind a LB VIP, configure them both to point to the
index on some shared storage (e.g. SAN), and count on the LB to fail-over to the
secondary master when the primary becomes unreachable?
If so, how do you deal with index locks?  You use the Native lock and count on
it disappearing when the primary master goes down?  That means you count on the
whole JVM process dying, which may not be the case...

* Or do you use tools like DRBD, Corosync, Pacemaker, etc. to keep 2 masters
with 2 separate indices in sync, while making sure you write to only 1 of them
via LB VIP or otherwise?

* Or ...


This thread is on a similar topic, but is inconclusive:
  http://search-lucene.com/m/aOsyN15f1qd1

Here is another similar thread, but this one doesn't cover how 2 masters are
kept in sync at all times:
  http://search-lucene.com/m/aOsyN15f1qd1

Thanks,
Otis
----
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
Lucene ecosystem search :: http://search-lucene.com/


Mime
View raw message