lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emir Arnautovic <emir.arnauto...@sematext.com>
Subject Re: Understanding solr commit
Date Mon, 25 Jan 2016 11:11:32 GMT
Hi Rahul,
If I got your mail right there is misconception of SolrCloud - nodes are 
infrastructure of cloud and collection is something that is "unit". So 
when you commit, you are committing changes you did on collection and 
SolrCloud will handle nodes. When you commit to three 3 nodes it is 
actually 3 commits to single collection.
It is not considered to be good practice to have script that does 
commits. Solr has autocommit functionality. You should also educate 
about soft v.s. hard commits. Following article is good starting point: 
https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/

Regards,
Emir

On 25.01.2016 12:02, Rahul Ramesh wrote:
> We are facing some issue and we are finding it difficult to debug the
> problem. We wanted to understand how solr commit works.
> A background on our setup:
> We have  3 Node Solr Cluster running in version 5.3.1. Its a index heavy
> use case. In peak load, we index 400-500 documents/second.
> We also want these documents to be visible as quickly as possible, hence we
> run an external script which commits every 3 mins.
>
> Consider the three nodes as N1, N2, N3. Commit is an synchronous operation.
> So, we will not get control till the commit operation is complete.
>
> Consider the following scenario. Although it looks like a basic scenario in
> distributed system:-) but we just wanted to eliminate this possibility.
>
> step 1 : At time T1, commit happens to Node N1
> step 2: At same time T1, we search for all the documents inserted in Node
> N2.
>
> My question is
>
> 1. Is commit an atomic operation? I mean, will commit happen on all the
> nodes at the same time?
> 2. Can we say that, the search result will always contain the documents
> before commit / or after commit . Or can it so happen that we get new
> documents fron N1, N2 but old documents (i.e., before commit)  from N3?
>
> Thank you,
> Rahul
>

-- 
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/


Mime
View raw message