lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nagendra Nagarajayya (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-3816) Need a more granular nrt system that is close to a realtime system.
Date Mon, 29 Oct 2012 13:38:13 GMT

    [ https://issues.apache.org/jira/browse/SOLR-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486012#comment-13486012
] 

Nagendra Nagarajayya commented on SOLR-3816:
--------------------------------------------

Lucene/Solr search and commit architecture is designed to work off a “point-in-time snapshots”
of the index. Any add/update/delete needs a commit to be visible to searches (or atleast a
soft-commit). soft-commit re-opens the SolrIndexSearcher object and can be a performance limitation
if the soft-commits happen more than one per second, see blog:http://searchhub.org/dev/2011/09/07/realtime-get/.
Realtime NRT  makes available a near realtime view of the index. So any changes made to the
index is immediately visible. Performance is not a limitation as it does not close the SolrIndexSearcher
object as with soft-commit.

Realtime NRT is also different from realtime-get which is a simple lookup by id and needs
the transaction log to be enabled. realtime-get does not have search capability. Realtime
NRT allows full search, so you could search by id, text, location, etc. using boolean, dismax,
faceting, range queries ie. no change to existing functionality. No new request handlers to
be defined in solrconfig.xml. So all of your existing queries work as it is with no changes,
except that the results returned are in near real time. Realtime NRT also does not need the
transaction update log needed by realtime-get. So you can turn this off for improved performance.
autoCommit freq can also be increased to an hour from the default of 15 secs for improved
performance (remember commits can slow down your update performance) 

More info about Realtime NRT and a integrated download of  Solr 4.0 with Realtime NRT is available
here:
http://solr-ra.tgels.org/realtime-nrt.jsp


(Attached a patch for Solr 4.0 release branch)
                
> Need a more granular nrt system that is close to a realtime system.
> -------------------------------------------------------------------
>
>                 Key: SOLR-3816
>                 URL: https://issues.apache.org/jira/browse/SOLR-3816
>             Project: Solr
>          Issue Type: Improvement
>          Components: clients - java, replication (java), search, SearchComponents - other,
SolrCloud, update
>    Affects Versions: 4.0
>            Reporter: Nagendra Nagarajayya
>              Labels: nrt, realtime, replication, search, solrcloud, update
>         Attachments: solr-3816-realtime_nrt.patch
>
>
> Need a more granular NRT system that is close to a realtime system. A realtime system
should be able to reflect changes to the index as and when docs are added/updated to the index.
soft-commit offers NRT and is more realtime friendly than hard commit but is limited by the
dependency on the SolrIndexSearcher being closed and reopened and offers a coarse granular
NRT. Closing and reopening of the SolrIndexSearcher may impact performance also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message