lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erick Erickson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-1028) Automatic core loading unloading for multicore
Date Mon, 29 Oct 2012 12:32:13 GMT

    [ https://issues.apache.org/jira/browse/SOLR-1028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485987#comment-13485987
] 

Erick Erickson commented on SOLR-1028:
--------------------------------------

Yeah, I've punted entirely on the zookeeper case so far, and I've been wondering about how
the two will come together. That said, I suspect SolrCloud will make use of most of this infrastructure
when they do come together, the use-case is how to provide a way to automatically load/unload
cores when you have a bunch of them resident but only want to have a subset "live" at any
one time. The use-case I'm working on here is on the order of 10,000 cores with, maybe, 100-200
active at any one time. My _really_ fuzzy conception is that if we can get SolrCloud to take
the place of the "CoreDescriptorProvider" mechanism in this kind of case, it'll "just work".
How all that plays with replicas/leaders/ZK the ZK election process and all of that is...er...not
clear to me...

FWIW, I hacked the code as an experiment to make it used in all the SolrCloud tests (without
the hack, anything that's ZK sensitive doesn't use this code path) and was pleasantly surprised
by how few of them failed, so I think reconciling the two ways of doing things won't be horrible.
Always room for surprises of course.

As you said, though, I suspect we'll be hashing this all out in the "how to bring this all
together" discussion, wherever we have it.
                
> Automatic core loading unloading for multicore
> ----------------------------------------------
>
>                 Key: SOLR-1028
>                 URL: https://issues.apache.org/jira/browse/SOLR-1028
>             Project: Solr
>          Issue Type: New Feature
>          Components: multicore
>            Reporter: Noble Paul
>            Assignee: Erick Erickson
>             Fix For: 4.1
>
>         Attachments: SOLR-1028.patch
>
>
> usecase: I have many small cores (say one per user) on a single Solr box . All the cores
are not be always needed . But when I need it I should be able to directly issue a search
request and the core must be STARTED automatically and the request must be served.
> This also requires that I must have an upper limit on the no:of cores that should be
loaded at any given point in time. If the limit is crossed the CoreContainer must unload a
core (preferably the least recently used core)  
> There must be a choice of specifying some cores as fixed. These cores must never be unloaded


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message