lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erick Erickson (JIRA)" <>
Subject [jira] [Commented] (SOLR-1306) Support pluggable persistence/loading of solr.xml details
Date Wed, 14 Nov 2012 12:42:13 GMT


Erick Erickson commented on SOLR-1306:

But isn't this just another CoreDescriptorProvider? Perhaps the default one? The way the code
works, you don't have to define _any_ cores in solr.xml, all you have to do is provide a class
for a pluggable descriptor provider. Unless I'm missing something, we could have a default
one that enumerated the cores tree and the stock Solr could ship without a solr.xml at all.
Maybe back up a step and have a custom CoreContainer instead/too that could be specified when
needed as a sysprop when starting Solr?  Or do we just need a few system properties rather
than a new class? That would go a long way towards simply nuking the need to ever have a solr.xml
at all.

And people who had other needs could _still_ do something other/more complex as needed.

Besides Paul's comment about 10s of K cores at the top level (and, Paul, I'm thinking of walking
a _tree_, so not all the top-levels of the cores would be in the same directory), there's
still the startup cost in the case I'm working on of enumerating, say, 15K cores on startup.
The pluggable core provider allows that cost to be amortized over some period of time and/or
fetched from a faster source.

This is outside the separate issue of lazy core loading, the lazy core cache, etc.

So I'm not sure I see any incompatibilities here now I've thought about it a bit more. In
fact, this patch seems like a step to accomplish that goal.
> Support pluggable persistence/loading of solr.xml details
> ---------------------------------------------------------
>                 Key: SOLR-1306
>                 URL:
>             Project: Solr
>          Issue Type: New Feature
>          Components: multicore
>            Reporter: Noble Paul
>            Assignee: Erick Erickson
>             Fix For: 4.1
>         Attachments: SOLR-1306.patch, SOLR-1306.patch, SOLR-1306.patch, SOLR-1306.patch
> Persisting and loading details from one xml is fine if the no:of cores are small and
the no:of cores are few/fixed . If there are 10's of thousands of cores in a single box adding
a new core (with persistent=true) becomes very expensive because every core creation has to
write this huge xml. 
> Moreover , there is a good chance that the file gets corrupted and all the cores become
unusable . In that case I would prefer it to be stored in a centralized DB which is backed
up/replicated and all the information is available in a centralized location. 
> We may need to refactor CoreContainer to have a pluggable implementation which can load/persist
the details . The default implementation should write/read from/to solr.xml . And the class
should be pluggable as follows in solr.xml
> {code:xml}
> <solr>
>   <dataProvider class="" attr1="val1" attr2="val2"/>
> </solr>
> {code}
> There will be a new interface (or abstract class ) called SolrDataProvider which this
class must implement

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message