lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Thacker (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SOLR-11807) Restore Collection should reuse maxShardsPerNode specified during backup
Date Wed, 20 Jun 2018 08:29:00 GMT

     [ https://issues.apache.org/jira/browse/SOLR-11807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Varun Thacker updated SOLR-11807:
---------------------------------
    Description: 
When you start Solr 6.6. and run the cloud example here's the log excerpt :
{code:java}
INFO  - 2018-06-20 13:44:47.452; org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider;
Cluster at localhost:9983 ready



Connecting to ZooKeeper at localhost:9983 ...

INFO  - 2018-06-20 13:44:47.491; org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider;
Cluster at localhost:9983 ready

Uploading /Users/varunthacker/Downloads/search-downloads/solr-6.6.0/server/solr/configsets/data_driven_schema_configs/conf
for config gettingstarted to ZooKeeper at localhost:9983



Creating new collection 'gettingstarted' using command:

http://localhost:8983/solr/admin/collections?action=CREATE&name=gettingstarted&numShards=2&replicationFactor=2&maxShardsPerNode=2&collection.configName=gettingstarted{code}
maxShardsPerNode get's set to 2 . 

 

Compare this to Solr 7.3 
{code:java}
INFO  - 2018-06-20 13:55:33.823; org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider;
Cluster at localhost:9983 ready

Created collection 'gettingstarted' with 2 shard(s), 2 replica(s) with config-set 'gettingstarted'{code}
So something changed and now we no longer set maxShardsPerNode and it defaults to -1 . 

 

-1 has special handing while creating a collection ( it means max int ) . This special handling
is not there while Restoring a collection and hence this fails

We should not set maxShardsPerNode to -1 in the first place

Steps to reproduce:
 # ./bin/solr start -e cloud -noprompt : This creates a 2 node cluster and a gettingstarted
collection which 2X2
2. Added 4 docs (id=1,2,3,4) with commit=true and openSearcher=true (default)
3. Call backup: http://localhost:8983/solr/admin/collections?action=BACKUP&name=gettingstarted_backup&collection=gettingstarted&location=/Users/varunthacker/solr-7.1.0
4. Call restore:
[http://localhost:8983/solr/admin/collections?action=restore&name=gettingstarted_backup&collection=restore_gettingstarted&location=/Users/varunthacker/solr-7.1.0]

  was:
Today we can never restore the cloud example gettingstarted collection because it sets maxShardsPerNode=-1
. We have special handling for -1 in CreateCollectionCmd but not in RestoreCmd.

Today RestoreCmd does has this check and it fails to restore the collection

{code}
    if ((numShards * totalReplicasPerShard) > (availableNodeCount * maxShardsPerNode))
{
      throw new SolrException(ErrorCode.BAD_REQUEST,
          String.format(Locale.ROOT, "Solr cloud with available number of nodes:%d is insufficient
for"
              + " restoring a collection with %d shards, total replicas per shard %d and maxShardsPerNode
%d."
              + " Consider increasing maxShardsPerNode value OR number of available nodes.",
              availableNodeCount, numShards, totalReplicasPerShard, maxShardsPerNode));
    }
{code}

Steps to reproduce:

Steps to reproduce:

1. ./bin/solr start -e cloud -noprompt : This creates a 2 node cluster and a gettingstarted
collection which 2X2
2. Added 4 docs (id=1,2,3,4) with commit=true and openSearcher=true (default)
3. Call backup: http://localhost:8983/solr/admin/collections?action=BACKUP&name=gettingstarted_backup&collection=gettingstarted&location=/Users/varunthacker/solr-7.1.0
4. Call restore:
http://localhost:8983/solr/admin/collections?action=restore&name=gettingstarted_backup&collection=restore_gettingstarted&location=/Users/varunthacker/solr-7.1.0


> Restore Collection should reuse maxShardsPerNode specified during backup
> ------------------------------------------------------------------------
>
>                 Key: SOLR-11807
>                 URL: https://issues.apache.org/jira/browse/SOLR-11807
>             Project: Solr
>          Issue Type: Bug
>      Security Level: Public(Default Security Level. Issues are Public) 
>          Components: Backup/Restore
>            Reporter: Varun Thacker
>            Assignee: Varun Thacker
>            Priority: Minor
>             Fix For: master (8.0), 7.5
>
>         Attachments: SOLR-11807.patch, SOLR-11807.patch
>
>
> When you start Solr 6.6. and run the cloud example here's the log excerpt :
> {code:java}
> INFO  - 2018-06-20 13:44:47.452; org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider;
Cluster at localhost:9983 ready
> Connecting to ZooKeeper at localhost:9983 ...
> INFO  - 2018-06-20 13:44:47.491; org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider;
Cluster at localhost:9983 ready
> Uploading /Users/varunthacker/Downloads/search-downloads/solr-6.6.0/server/solr/configsets/data_driven_schema_configs/conf
for config gettingstarted to ZooKeeper at localhost:9983
> Creating new collection 'gettingstarted' using command:
> http://localhost:8983/solr/admin/collections?action=CREATE&name=gettingstarted&numShards=2&replicationFactor=2&maxShardsPerNode=2&collection.configName=gettingstarted{code}
> maxShardsPerNode get's set to 2 . 
>  
> Compare this to Solr 7.3 
> {code:java}
> INFO  - 2018-06-20 13:55:33.823; org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider;
Cluster at localhost:9983 ready
> Created collection 'gettingstarted' with 2 shard(s), 2 replica(s) with config-set 'gettingstarted'{code}
> So something changed and now we no longer set maxShardsPerNode and it defaults to -1
. 
>  
> -1 has special handing while creating a collection ( it means max int ) . This special
handling is not there while Restoring a collection and hence this fails
> We should not set maxShardsPerNode to -1 in the first place
> Steps to reproduce:
>  # ./bin/solr start -e cloud -noprompt : This creates a 2 node cluster and a gettingstarted
collection which 2X2
> 2. Added 4 docs (id=1,2,3,4) with commit=true and openSearcher=true (default)
> 3. Call backup: http://localhost:8983/solr/admin/collections?action=BACKUP&name=gettingstarted_backup&collection=gettingstarted&location=/Users/varunthacker/solr-7.1.0
> 4. Call restore:
> [http://localhost:8983/solr/admin/collections?action=restore&name=gettingstarted_backup&collection=restore_gettingstarted&location=/Users/varunthacker/solr-7.1.0]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message