lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jack Krupansky" <j...@basetechnology.com>
Subject Re: Committing when indexing in parallel
Date Sun, 15 Sep 2013 04:38:28 GMT
You really are barking up the wrong tree here. Solr is a search engine, 
designed for batch update and "eventual consistency". This fantasy you have 
of knowing exactly when a document is committed is completely inappropriate 
with Solr. Sure, you can in fact do a hard commit at any time to guarantee 
that recent updates are immediately searchable, but that is strongly 
discouraged for performance reasons, since Solr is batch update oriented.

And the level of detail you are requesting is merely how Solr happens to 
work today and is not necessarily guaranteed for future releases - since the 
guaranteed model is only for commits with eventual consistency.

It appears that you are trying to imagine Solr as a traditional, 
transaction-based database, when that is not the case.

I've asked you before to disclose what problem you are really trying to 
solve, and so far you have not yet let us in on your secret. You are 
certainly welcome to peruse the Solr and Lucene source code if your goal is 
merely idle curiosity, but you really should not be designing a Solr-based 
application around this non-guaranteed level of detail.

Soft commit, commit within, hard commit, real time get, and eventual 
consistency are the proper tools to design a Solr-based application around.

And if you wish to have multiple, uncoordinated update streams, you need to 
relax your requirements for eventual consistency even further.

In short, to summarize again, Solr is not a transaction-based database, but 
instead is a batch-oriented search engine with eventual consistency. Focus 
on exploiting Solr's strengths, not trying to treat Solr as something that 
it is not.

-- Jack Krupansky

-----Original Message----- 
From: Phani Chaitanya
Sent: Saturday, September 14, 2013 7:36 PM
To: solr-user@lucene.apache.org
Subject: Re: Committing when indexing in parallel

Thanks Erick. I completely did not get the point you are trying to make 
w.r.t
my question. I'll add it again according to your example.

In the example you gave w.r.t P1.1, P2.1, P1.2, P2.2 and P1 issues a commit,
I understand that all documents are committed.

Now what happens when P1 issues a commit and P2 sends another document, say
P2.3, to index at the same time ?

As you said the requests are treated as if they are serial - if P1.commit is
the first one among P1.commit & P2.3, P2.3 will be indexed into a new
segment ?

Regards,
Phani.



-----
Phani Chaitanya
--
View this message in context: 
http://lucene.472066.n3.nabble.com/Committing-when-indexing-in-parallel-tp4089953p4090120.html
Sent from the Solr - User mailing list archive at Nabble.com. 


Mime
View raw message