lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From André Bois-Crettez (JIRA) <j...@apache.org>
Subject [jira] [Commented] (SOLR-3866) CoreAdmin SWAP and RENAME need fixed/defined when using SolrCloud
Date Wed, 05 Dec 2012 15:12:58 GMT

    [ https://issues.apache.org/jira/browse/SOLR-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510524#comment-13510524
] 

André Bois-Crettez commented on SOLR-3866:
------------------------------------------

We have a similar use case too, detailed here http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201212.mbox/%3C50BDF33C.6060203%40kelkoo.com%3E
I understand that CoreAdmin SWAP should first be made to work consistently with ZooKeeper,
but then using the collections API for a SWAP rather than lower level CoreAdmin would be even
simpler to use.


Assumptions:
 * 'collection_main' and 'collection_temp' are configured in the same way, but can contain
different documents.
For example, each solr instance in the SolrCloud have both main and temp collections, sharding
is done the same way, etc.

 * It is not a problem if the SWAP can take some time to be taken into account across all
the solr instances.
Until the whole SolrCloud has swapped, some queries may show some docs with the old main index,
and some docs with the new main, but this is acceptable.


So the ideal scenario would be :

we would like that search clients always query the main collection :
http://hostname:8983/solr/collection_main/select?...

1) on our indexing process, we nominally update the same main collection :
http://hostname:8983/solr/collection_main/update?...

*but* when we need a full recovery of the index, ie. remove every document and re-index everything,
we can not do that on collection_main as search clients will miss documents until the recovery
is completed.

2) So during a recovery, the indexing process will now work on a *temp* collection :
http://hostname:8983/solr/collection_temp/update?... <delete><query>*:*</query></delete>
http://hostname:8983/solr/collection_temp/update?... <add><doc> ....</add>
http://hostname:8983/solr/collection_temp/update?... <commit/>

3) When we are ready to make this new index available to search clients, the indexing process
could trigger this with a swap :
http://hostname:8983/solr/admin/collections?action=SWAP&name=collection_temp&other=collection_main

Now we are back to the situation in 1)



Currently in solr-4.0.0, a CoreAdmin SWAP messes the clusterstate in Zk (clearly visible in
a setup with 2 shards and 2 replicas), how can we move toward a solution ?
Please ask if more information is needed.
                
> CoreAdmin SWAP and RENAME need fixed/defined when using SolrCloud
> -----------------------------------------------------------------
>
>                 Key: SOLR-3866
>                 URL: https://issues.apache.org/jira/browse/SOLR-3866
>             Project: Solr
>          Issue Type: Bug
>    Affects Versions: 4.0-ALPHA, 4.0-BETA, 4.0
>            Reporter: Hoss Man
>
> We need to define what the expected behavior of using the CoreAdminHandler's SWAP and
RENAME commands is if you are running in SolrCloud mode.
> At the moment, it seems to introduce a disconnect between which "collection" the SolrCore
thinks it's a part of (and what appears in the persisted solr.xml) vs what collection ZooKeeper
thinks the SolrCore(s) that were swaped/renamed are a part of.  We should either "fix" this,
or document it if it as actually consistent and intentional for low level controls, or disable
commands like SWAP/RENAME in SolrCloud mode
> https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201209.mbox/%3CCALB4QrP2GZAwLeAiy%3DfcmOLYbc5r0i9Tp1DquyPS8mMJPwCgnw%40mail.gmail.com%3E

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message