lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erick Erickson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-1293) Support for large no:of cores and faster loading/unloading of cores
Date Sat, 27 Oct 2012 14:07:14 GMT

    [ https://issues.apache.org/jira/browse/SOLR-1293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485442#comment-13485442
] 

Erick Erickson commented on SOLR-1293:
--------------------------------------

About persistence. And about coreDescriptorProvider in general. If one is supplied, I'm thinking
that it will always have first crack at most things having to do with getting a CoreDescriptor.
For instance:
1> When persisting a core, ask the coreDescriptor whether to persist to solr.xml.
2> When listing cores, give preference to any descriptor the coreDescriptor knows about.
I.e. override the ones in any CoreContainer lists with ones from the provider.
3> ???

The mind-set here is that, if a CoreDescriptorProvider is present, it should be the arbiter
of relevant decisions about that core. We can default to reasonable stuff (e.g. default behavior
for CoreDescriptorProvider.shouldPersist(String coreName) is to return false)

I'm seeing one other thing related to persistence, NOT having to do with a CoreDescriptorProvider.
Since CoreContainer now has two new params, loadOnStartup=true and swappable=false magically
show up in the persisted file if they aren't specified. It would be a bit more aesthetic to
only show what was specified by the user, but I'm not sure it's worth any effort, and it appears
that this is true for some other properties as well.
                
> Support for large no:of cores and faster loading/unloading of cores
> -------------------------------------------------------------------
>
>                 Key: SOLR-1293
>                 URL: https://issues.apache.org/jira/browse/SOLR-1293
>             Project: Solr
>          Issue Type: New Feature
>          Components: multicore
>            Reporter: Noble Paul
>            Assignee: Erick Erickson
>             Fix For: 4.1
>
>         Attachments: SOLR-1293.patch
>
>
> Solr , currently ,is not very suitable for a large no:of homogeneous cores where you
require fast/frequent loading/unloading of cores . usually a core is required to be loaded
just to fire a search query or to just index one document
> The requirements of such a system are.
> * Very efficient loading of cores . Solr cannot afford to read and parse and create Schema,
SolrConfig Objects for each core each time the core has to be loaded ( SOLR-919 , SOLR-920)
> * START STOP core . Currently it is only possible to unload a core (SOLR-880)
> * Automatic loading of cores . If a core is present and it is not loaded and a request
comes for that load it automatically before serving up a request
> * As there are a large no:of cores , all the cores cannot be kept loaded always. There
has to be an upper limit beyond which we need to unload a few cores (probably the least recently
used ones)
> * Automatic allotment of dataDir for cores. If the no:of cores is too high al the cores'
dataDirs cannot live in the same dir. There is an upper limit on the no:of dirs you can create
in a unix dir w/o affecting performance

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message