lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "olivier soyez (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-1575) loading solr.xml takes a lot of time when there are lot of cores
Date Thu, 06 Dec 2012 17:35:09 GMT

    [ https://issues.apache.org/jira/browse/SOLR-1575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511545#comment-13511545
] 

olivier soyez commented on SOLR-1575:
-------------------------------------

In fact, I used the 4.0 code + SOLR-1293 modified code to test lotsofcores solr 4.0 (to be
able to handle 50k cores per server with 1000 lives cores max at any time, shared configs,
numBuckets, lazy load cores, efficient start cores, automatic start/stop cores, core creation
on the fly, efficient fresh start). It's the reason why I have the loadOnStart="false" option.

Anyways, I pull down 5x and without changes, I try to start the server (50K cores with "loadOnStartup=false"),
and same issue (very very long time to start ...)
After that, I applied the change (line 590 of CoreContainer.java Apache Solr 5.0 trunck).
In this way, the node you are working with is detached from the main document. So, it's a
huge speed up for the evaluate method used in readProperties function called for each core,
especially in case of lots of cores. And now it takes a few seconds to start the server.

Thanks, I would following SOLR-4083, because in your specific use case (50k cores), deprecate
solr.xml entirely could be a scalability problem (especially fresh start time).
                
> loading solr.xml takes a lot of time when there are lot of cores
> ----------------------------------------------------------------
>
>                 Key: SOLR-1575
>                 URL: https://issues.apache.org/jira/browse/SOLR-1575
>             Project: Solr
>          Issue Type: Improvement
>          Components: multicore
>            Reporter: Noble Paul
>            Priority: Minor
>             Fix For: 1.5
>
>
> We have around 38K cores which takes up around 25 mins to start the server. Keep in mind
that no cores are started and this is the time taken for loading the solr.xml.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message