lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Noble Paul നോബിള്‍ नोब्ळ् <noble.p...@corp.aol.com>
Subject Re: Add replication push support?
Date Thu, 23 Jul 2009 12:52:05 GMT
On Thu, Jul 23, 2009 at 5:58 PM, Grant Ingersoll<gsingers@apache.org> wrote:
> If you bring into play the ZooKeeper work I started, then I think this
> becomes easier.  Masters can put out notifications on ZooKeeper that slaves
> can subscribe to and everyone knows when everyone is up and happy.
+1
w/o a central repository (like zk) this is going to be tricky
>
> -Grant
>
>
> On Jul 23, 2009, at 1:18 AM, Noble Paul നോബിള്‍ नोब्ळ् wrote:
>
>> There is an issue already created for realtime replication
>> https://issues.apache.org/jira/browse/SOLR-982
>>
>> In general push based replication is more error prone because the
>> master has to be aware of the state of each slave . it is difficault
>> because a slave may go down at any time and may come up later or new
>> slaves get added randomly. In a pull based replication the master is
>> agnostic of the slaves and a slave can easily know his current state
>> and take appropriate action.
>>
>> We can take a middle path. The slaves can register with the master for
>> notifications for availability of new snapshots. This can be as good
>> as a push based replication without the complexities associated with
>> it.
>> I have raised as issue already
>> https://issues.apache.org/jira/browse/SOLR-1305
>>
>> On Thu, Jul 23, 2009 at 2:58 AM, Jason
>> Rutherglen<jason.rutherglen@gmail.com> wrote:
>>>
>>> It would be useful to push snapshots as they are created from
>>> the master to the slaves. I prefer this approach to constant
>>> polling by slaves. Partially because the timing could be off on
>>> the slave servers, the data is replicated, and the user sees
>>> different snapshots?
>>>
>>> Something like a virtual 2 phase commit, where phase 1 is
>>> replicate the new snapshots and load the searcher, phase two is
>>> all slaves synchronously expose the new searcher. I'm also
>>> wondering how we'll handle synchronous slaves with near realtime
>>> search.
>>>
>>
>>
>>
>> --
>> -----------------------------------------------------
>> Noble Paul | Principal Engineer| AOL | http://aol.com
>
> --------------------------
> Grant Ingersoll
> http://www.lucidimagination.com/
>
> Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using
> Solr/Lucene:
> http://www.lucidimagination.com/search
>
>



-- 
-----------------------------------------------------
Noble Paul | Principal Engineer| AOL | http://aol.com

Mime
View raw message