lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Otis Gospodnetic (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-1293) Support for large no:of cores and faster loading/unloading of cores
Date Tue, 23 Oct 2012 15:33:17 GMT

    [ https://issues.apache.org/jira/browse/SOLR-1293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13482413#comment-13482413
] 

Otis Gospodnetic commented on SOLR-1293:
----------------------------------------

General comment:
We may want the index/core re-opener to remain aware of previous locations (nodes) on which
cores were opened for the purposes of reusing any possible OS-level caches that may still
exist on those nodes for that core.  For example, if the cluster has nodes 1-100 and core
Foo was on nodes 1, 2, and 3 before it was closed, then maybe next time it needs to be opened
it would ideally be opened on those 1, 2, and 3 nodes.  Of course, nodes 1, 2, or 3 may no
longer be around or may be currently overloaded, or.... in which case alternative nodes need
to be picked.

                
> Support for large no:of cores and faster loading/unloading of cores
> -------------------------------------------------------------------
>
>                 Key: SOLR-1293
>                 URL: https://issues.apache.org/jira/browse/SOLR-1293
>             Project: Solr
>          Issue Type: New Feature
>          Components: multicore
>            Reporter: Noble Paul
>             Fix For: 4.1
>
>         Attachments: SOLR-1293.patch
>
>
> Solr , currently ,is not very suitable for a large no:of homogeneous cores where you
require fast/frequent loading/unloading of cores . usually a core is required to be loaded
just to fire a search query or to just index one document
> The requirements of such a system are.
> * Very efficient loading of cores . Solr cannot afford to read and parse and create Schema,
SolrConfig Objects for each core each time the core has to be loaded ( SOLR-919 , SOLR-920)
> * START STOP core . Currently it is only possible to unload a core (SOLR-880)
> * Automatic loading of cores . If a core is present and it is not loaded and a request
comes for that load it automatically before serving up a request
> * As there are a large no:of cores , all the cores cannot be kept loaded always. There
has to be an upper limit beyond which we need to unload a few cores (probably the least recently
used ones)
> * Automatic allotment of dataDir for cores. If the no:of cores is too high al the cores'
dataDirs cannot live in the same dir. There is an upper limit on the no:of dirs you can create
in a unix dir w/o affecting performance

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message