lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gerald Reinhart <gerald.reinh...@kelkoo.com>
Subject Re: [Migration Solr4 to Solr5] Collection reload error
Date Mon, 07 Mar 2016 11:11:04 GMT

Hi,

      To give you some context, we are migrating from Solr4 and solr5,
the client code and the configuration haven't changed but now we are
facing this problem. We have already checked the commit behaviour
configuration and it seems good.

Here it is :

Server side, we have 2 collections (main and temp with blue and green
aliases) :

    solrconfig.xml:

   <!-- The default high-performance update handler -->
   <updateHandler class="solr.DirectUpdateHandler2">
      (...)
      <autoCommit>
        <maxTime>900000</maxTime>
        <openSearcher>false</openSearcher>
      </autoCommit>

      <!-- Soft commit: commented => not used   -->

   </updateHandler>

Client side, we have 2 different modes:

     1 - Full recovery :

             - Delete all documents from the temp collection
               solrClient.deleteByQuery("*:*")

             - Add all new documents in temp collection (can be more
than 5Millions),
               solrClient.add(doc, -1) // commitWithinMs == -1

             -  Commit when all documents are added
               solrClient.commit(false,false) // waitFlush == false ,
waitSearcher == false

             -  Swap blue and green using "create alias" command

             -  Reload the temp collection to clean the cache. This is
at this point we have the issue.

     2 - Incremental :

             -  Add or delete documents from the main collection
                solrClient.add(doc, 1800000)           // commitWithin
== 30 mn
                solrClient.deleteById(doc, 1800000) // commitWithin == 30 mn

Maybe you will spot something obviously wrong ?

Thanks

Gérald and Elodie



On 03/04/2016 12:41 PM, Dmitry Kan wrote:
> Hi,
>
> Check the the autoCommit and autoSoftCommit nodes in the solrconfig.xml.
> Set them to reasonable values. The idea is that if you commit too often,
> searchers will be warmed up and thrown away. If at any point in time you
> get overlapping commits, there will be several searchers sitting on the
> deck.
>
> Dmitry
>
> On Mon, Feb 29, 2016 at 4:20 PM, Gerald Reinhart <gerald.reinhart@kelkoo.com
>> wrote:
>> Hi,
>>
>>     We are facing an issue during a migration from Solr4 to Solr5.
>>
>> Given
>>     - migration from solr 4.10.4 to 5.4.1
>>     - 2 collections
>>     - cloud with one leader and several replicas
>>     - in solrconfig.xml: maxWarmingSearchers=1
>>     - no code change
>>
>> When collection reload using /admin/collections using solrj
>>
>> Then
>>
>> 2016-02-29 13:42:49,011 [http-8080-3] INFO
>> org.apache.solr.core.CoreContainer:reload:848  - Reloading SolrCore
>> 'fr_blue' using configuration from collection fr_blue
>> 2016-02-29 13:42:45,428 [http-8080-6] INFO
>> org.apache.solr.search.SolrIndexSearcher:<init>:237  - Opening
>> Searcher@58b65fc[fr_blue] main
>> (...)
>> 2016-02-29 13:42:49,077 [http-8080-3] WARN
>> org.apache.solr.core.SolrCore:getSearcher:1762  - [fr_blue] Error
>> opening new searcher. exceeded limit of maxWarmingSearchers=1, try again
>> later.
>> 2016-02-29 13:42:49,091 [http-8080-3] ERROR
>> org.apache.solr.handler.RequestHandlerBase:log:139  -
>> org.apache.solr.common.SolrException: Error handling 'reload' action
>>          at
>>
>> org.apache.solr.handler.admin.CoreAdminHandler.handleReloadAction(CoreAdminHandler.java:770)
>>          at
>>
>> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:230)
>>          at
>>
>> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:184)
>>          at
>>
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
>>          at
>>
>> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:664)
>>          at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:438)
>>          at
>>
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:223)
>>          at
>>
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:181)
>>          at
>>
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
>>          at
>>
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>>          at
>>
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
>>          at
>>
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
>>          at
>>
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
>>          at
>>
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>>          at
>> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
>>          at
>>
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>>          at
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
>>          at
>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
>>          at
>>
>> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
>>          at
>> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
>>          at java.lang.Thread.run(Thread.java:745)
>> Caused by: org.apache.solr.common.SolrException: Unable to reload core
>> [fr_blue]
>>          at
>> org.apache.solr.core.CoreContainer.reload(CoreContainer.java:854)
>>          at
>>
>> org.apache.solr.handler.admin.CoreAdminHandler.handleReloadAction(CoreAdminHandler.java:768)
>>          ... 20 more
>> Caused by: org.apache.solr.common.SolrException: Error opening new
>> searcher. exceeded limit of maxWarmingSearchers=1, try again later.
>>          at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1764)
>>          at org.apache.solr.core.SolrCore.reload(SolrCore.java:474)
>>          at
>> org.apache.solr.core.CoreContainer.reload(CoreContainer.java:849)
>>          ... 21 more
>>
>>
>> Thanks
>>
>>
>> Gérald and Elodie
>>
>>
>> Kelkoo SAS
>> Société par Actions Simplifiée
>> Au capital de € 4.168.964,30
>> Siège social : 158 Ter Rue du Temple 75003 Paris
>> 425 093 069 RCS Paris
>>
>> Ce message et les pièces jointes sont confidentiels et établis à
>> l'attention exclusive de leurs destinataires. Si vous n'êtes pas le
>> destinataire de ce message, merci de le détruire et d'en avertir
>> l'expéditeur.
>>
>
>


--

Kelkoo



*Gérald Reinhart *Software engineer

*E*gerald.reinhart@kelkoo.com <mailto:steve.conway@kelkoo.com>
*Y!Messenger*gerald.reinhart

*A*Rue des Méridiens 38130 Echirolles






Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention exclusive
de leurs destinataires. Si vous n'êtes pas le destinataire de ce message, merci de le détruire
et d'en avertir l'expéditeur.

Mime
View raw message