lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <markrmil...@gmail.com>
Subject Re: NRT consistency
Date Tue, 12 Apr 2011 05:00:10 GMT

On Apr 11, 2011, at 2:41 PM, Otis Gospodnetic wrote:

> I think what's being described here is a lot like what I *think* ElasticSearch 
> does, where there is no single master and index changed made to any node get 
> propagated to N-1 other nodes (N=number of index replicas).  I'm not sure how it 
> deals with situations where "incompatible" index changes are made to the same 
> index via 2 different nodes at the same time.  Is that what vector clocks are 
> about?

Right - you have to have some sort of conflict detection/resolution - Amazon Dynamo uses vector
clocks for this.

> 
> Otis
> ----
> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
> Lucene ecosystem search :: http://search-lucene.com/
> 
> 
> 
> ----- Original Message ----
>> From: Mark Miller <markrmiller@gmail.com>
>> To: java-user@lucene.apache.org
>> Sent: Mon, April 11, 2011 11:52:05 AM
>> Subject: Re: NRT consistency
>> 
>> 
>> On Apr 10, 2011, at 4:34 AM, Em wrote:
>> 
>>> Hello list,
>>> 
>>> I am currently trying to understand Lucene's Near-Real-Time-Feature  which
>>> was covered in "Lucene in Action, Second Edition".
>>> 
>>> Let's say I got a distributed system with a master and a slave.
>>> 
>>> In Solr replication is solved by checking for any differences in  the
>>> index-directory and to consume those differences to keep indices  
> consistent.
>>> 
>>> How is this possible within a NRT-System? Is there  any possibility to
>>> consume snapshots of the internal buffer of the index  writer to send them 
> to
>>> the slave?
>> 
>> I think for near real time,  Solr index replication may not be appropriate. 
>> Though I think it would be cool  to use Andrzej's mythical single pass index 
>> splitter to create a single+ doc  segment that could be shipped around.
>> 
>> Most likely, a system that just  sends each doc to each replica is probably 
>> going to work a lot better.  Introduces other issues of course - some of which 
>> we hope to alleviate with  further SolrCloud work.
>> 
>>> 
>>> Regards,
>>> Em
>>> 
>>> --
>>> View this message in context: 
>> http://lucene.472066.n3.nabble.com/NRT-consistency-tp2801878p2801878.html
>>> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>> 
>>> ---------------------------------------------------------------------
>>> To  unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>> 
>> 
>> - Mark Miller
>> lucidimagination.com
>> 
>> Lucene/Solr User  Conference
>> May 25-26, San  Francisco
>> www.lucenerevolution.org
>> 
>> 
>> 
>> 
>> 
>> 
>> ---------------------------------------------------------------------
>> To  unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For  additional commands, e-mail: java-user-help@lucene.apache.org
>> 
>> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 

- Mark Miller
lucidimagination.com

Lucene/Solr User Conference
May 25-26, San Francisco
www.lucenerevolution.org






---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message