lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nagendra Nagarajayya (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-3816) Need a more granular nrt system that is close to a realtime system.
Date Fri, 09 Nov 2012 16:38:14 GMT

    [ https://issues.apache.org/jira/browse/SOLR-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494105#comment-13494105
] 

Nagendra Nagarajayya commented on SOLR-3816:
--------------------------------------------

@Otis:

Yes, you could set to something low or 0, but this means it has to close and open the SolrIndexSearcher
this often. SolrIndexSearcher is a heavy object that is reference counted so there may be
searches going on, etc. has lots critical areas that need to be synchronized to close and
reopen a new searcher, warms it up, etc.; was not meant for this kind of a use ...

Realtime-search just gets a new nrt reader from the writer and passes this along to the Searcher,
a lean searcher with no state. In the future if lucene's developers make the reader more realtime
so it sees more changes as they happen at the writer realtime-search should be able to handle
it ...

"Quote from the user using realtime-search"
Insertion speed – while we can’t really explain this, we are able to insert 70k records
per second at a steady rate over time with RA, while we can only do 40k at a descending rate
with normal Solr. Granted we haven’t even slightly configured regular Solr for high speed
insertion with regard to segment configs, but this was good for us to get us quickly off the
ground.
"end quote"

I think has gotten better with the 4.0 release. I have also requested the user to benchmark
and update the JIRA as I don't have the required hardware.


                
> Need a more granular nrt system that is close to a realtime system.
> -------------------------------------------------------------------
>
>                 Key: SOLR-3816
>                 URL: https://issues.apache.org/jira/browse/SOLR-3816
>             Project: Solr
>          Issue Type: Improvement
>          Components: clients - java, replication (java), search, SearchComponents - other,
SolrCloud, update
>    Affects Versions: 4.0
>            Reporter: Nagendra Nagarajayya
>              Labels: nrt, realtime, replication, search, solrcloud, update
>         Attachments: alltests_passed_with_realtime_turnedoff.log, SOLR-3816_4.0_branch.patch,
SOLR-3816-4.x.trunk.patch, solr-3816-realtime_nrt.patch
>
>
> Need a more granular NRT system that is close to a realtime system. A realtime system
should be able to reflect changes to the index as and when docs are added/updated to the index.
soft-commit offers NRT and is more realtime friendly than hard commit but is limited by the
dependency on the SolrIndexSearcher being closed and reopened and offers a coarse granular
NRT. Closing and reopening of the SolrIndexSearcher may impact performance also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message