lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <apa...@elyograg.org>
Subject Re: SOLR nrt read writes
Date Tue, 14 Jul 2015 22:03:31 GMT
On 7/14/2015 12:19 PM, Bhawna Asnani wrote:
> I have a use case where we have to write data into solr and immediately
> read it back.
> The read is not get by Id but a search call.
>
> I am doing a softCommit after every such write which needs to be visible
> immediately.
> However sometimes the changes are not visible immediately.
>
> We have a solr cloud but I have also tried sending reads, writes and
> commits to cloud leader only and still there is some latency.
>
> Has anybody tried to use solr this way?

Don't ignore what Erick has said just because you're getting this reply
from someone else.  That advice is correct.  My intent here is to
provide more detail.

Since you are not doing a retrieval by id (uniqueKey field), you can't
use the Realtime Get handler.  That handler would get the latest version
of a doc, whether it has been committed or not.  The transaction logs
(configured with updateLog in solrconfig.xml) are used to retrieve
uncommitted information.  Can you change your retrieval so it's by id
rather than a search query?  If you can, that might solve this for you.

Normally, if you do a commit operation with openSearcher=true and
waitSearcher=true, control of the program will not be returned until
that commit is completely done ... but as Erick said, if you are doing a
LOT of commits very quickly, you're probably going to exceed
maxWarmingSearchers, and in that scenario, you cannot rely on using the
commit operation as a blocker for your retrieval attempt.

In order to have any hope of getting what you want with your current
methods, your commit frequency must be low enough that each commit has
time to finish before the next one begins.  I personally would not do
commits more often than once a minute.  Commits on my larger index
shards are known to take up to ten seconds when the index is quiet, and
even more if the index is busy.  There are ways to make commits happen
faster, but it often involves disabling features that you might want to
leave enabled.

Thanks,
Shawn


Mime
View raw message