lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From primoz.sk...@policija.si
Subject Re: Regarding Solr Cloud issue...
Date Wed, 16 Oct 2013 08:49:07 GMT
Yap, you are right - I only created extra replicas with cores API. For a 
new shard I had to use "split shard" command.

My apologies.

Primo┼ż



From:   Shalin Shekhar Mangar <shalinmangar@gmail.com>
To:     solr-user@lucene.apache.org
Date:   16.10.2013 10:45
Subject:        Re: Regarding Solr Cloud issue...



If the initial collection was created with a numShards parameter (and 
hence
compositeId router then there was no way to create a new logical shard. 
You
can add replicas with the core admin API but only to shards that already
exist. A new logical shard can only be created by splitting an existing 
one.

The "createshard" API also has the same limitation -- it cannot create a
shard for a collection with compositeId router. It is supposed to be used
for collections with custom sharding (i.e. "implicit" router). In such
collections, there is no concept of a hash range and routing is done
explicitly by the user using the "shards" parameter in the request or by
sending the request to the target core/node directly.

So, in summary, attempting to add a new logical shard to a collection with
compositeId router via CoreAdmin APIs is wrong, unsupported and should be
disallowed. Adding replicas to existing logical shards is okay though.


On Wed, Oct 16, 2013 at 12:56 PM, <primoz.skale@policija.si> wrote:

> If I am not mistaken the only way to create a new shard from a 
collection
> in 4.4.0 was to use cores API. That worked fine for me until I used
> *other* cores API commands. Those usually produced null ranges.
>
> In 4.5.0 this is fixed with newly added commands "createshard" etc. to 
the
> collections API, right?
>
> Primoz
>
>
>
> From:   Shalin Shekhar Mangar <shalinmangar@gmail.com>
> To:     solr-user@lucene.apache.org
> Date:   16.10.2013 09:06
> Subject:        Re: Regarding Solr Cloud issue...
>
>
>
> Chris, can you post your complete clusterstate.json? Do all shards have 
a
> null range? Also, did you issue any core admin CREATE commands apart 
from
> the create collection api.
>
> Primoz, I was able to reproduce this but by doing an illegal operation.
> Suppose I create a collection with numShards=5 and then I issue a core
> admin create command such as:
>
> 
http://localhost:8983/solr/admin/cores?action=CREATE&name=xyz&collection=mycollection51&shard=shard6

>
>
> Then a "shard6" is added to the collection with a null range. This is a
> bug
> because we should never allow such a core admin create to succeed 
anyway.
> I'll open an issue.
>
>
>
> On Wed, Oct 16, 2013 at 11:49 AM, <primoz.skale@policija.si> wrote:
>
> > I sometimes also do get null ranges when doing colletions/cores API
> > actions CREATE or/and UNLOAD, etc... In 4.4.0 that was not easily 
fixed
> > because zkCli had problems with "putfile" command, but in 4.5.0 it 
works
> > OK. All you have to do is "download" clusterstate.json from ZK ("get
> > /clusterstate.json"), fix ranges to appropriate values and upload the
> file
> > back to ZK with zkCli.
> >
> > But why those null ranges happen at all is beyond me :)
> >
> > Primoz
> >
> >
> >
> > From:   Shalin Shekhar Mangar <shalinmangar@gmail.com>
> > To:     solr-user@lucene.apache.org
> > Date:   16.10.2013 07:37
> > Subject:        Re: Regarding Solr Cloud issue...
> >
> >
> >
> > I'm sorry I am not able to reproduce this issue.
> >
> > I started 5 solr-4.4 instances.
> > I copied example directory into example1, example2, example3 and
> example4
> > cd example; java -Dbootstrap_confdir=./solr/collection1/conf
> > -Dcollection.configName=myconf -DzkRun -DnumShards=1 -jar start.jar
> > cd example1; java -Djetty.port=7574 -DzkHost=localhost:9983 -jar
> start.jar
> > cd example2; java -Djetty.port=7575 -DzkHost=localhost:9983 -jar
> start.jar
> > cd example3; java -Djetty.port=7576 -DzkHost=localhost:9983 -jar
> start.jar
> > cd example4; java -Djetty.port=7577 -DzkHost=localhost:9983 -jar
> start.jar
> >
> > After that I invoked:
> >
> >
>
> 
http://localhost:8983/solr/admin/collections?action=CREATE&name=mycollection51&numShards=5&replicationFactor=1

>
> >
> >
> > I can see all shards having non-null ranges in clusterstate.
> >
> >
> > On Tue, Oct 15, 2013 at 8:47 PM, Chris <christudas@gmail.com> wrote:
> >
> > > Hi Shalin,.
> > >
> > > Thank you for your quick reply. I appreciate all the help.
> > >
> > > I started the solr cloud servers first...with 5 nodes.
> > >
> > > then i issued a command like below to create the shards -
> > >
> > >
> > >
> >
> >
>
> 
http://localhost:8983/solr/admin/collections?action=CREATE&name=mycollection&numShards=5&replicationFactor=1

>
> >
> > > <
> > >
> >
> >
>
> 
http://localhost:8983/solr/admin/collections?action=CREATE&name=mycollection&numShards=3&replicationFactor=4

>
> >
> > > >
> > >
> > > Please advice.
> > >
> > > Regards,
> > > Chris
> > >
> > >
> > > On Tue, Oct 15, 2013 at 8:07 PM, Shalin Shekhar Mangar <
> > > shalinmangar@gmail.com> wrote:
> > >
> > > > How did you create these shards? Can you tell us how to reproduce
> the
> > > > issue?
> > > >
> > > > Any shard in a collection with compositeId router should never 
have
> > null
> > > > ranges.
> > > >
> > > >
> > > > On Tue, Oct 15, 2013 at 7:07 PM, Chris <christudas@gmail.com> 
wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I am using solr 4.4 as cloud. while creating shards i see that 
the
> > last
> > > > > shard has range of "null". i am not sure if this is a bug.
> > > > >
> > > > > I am stuck with having null value for the range in
> clusterstate.json
> > > > > (attached below)
> > > > >
> > > > > "shard5":{ "range":null, "state":"active",
> > "replicas":{"core_node1":{
> > > > > "state":"active", "core":"Web_shard5_replica1",
> > > > > "node_name":"domain-name.com:1981_solr", "base_url":"
> > > > > http://domain-name.com:1981/solr", "leader":"true"}}}},
> > > > > "router":"compositeId"},
> > > > >
> > > > > I tried to use zookeeper cli to change this, but it was not able
> to.
> > I
> > > > > tried to locate this file, but didn't find it anywhere.
> > > > >
> > > > > Can you please let me know how do i change the range from null 
to
> > > > something
> > > > > meaningful? i have the range that i need, so if i can find the
> file,
> > > > maybe
> > > > > i can change it manually.
> > > > >
> > > > > My next question is - can we have a catch all for ranges, i mean
> if
> > > > things
> > > > > don't match any other range then insert in this shard..is this
> > > possible?
> > > > >
> > > > > Kindly advice.
> > > > > Chris
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > > Shalin Shekhar Mangar.
> > > >
> > >
> >
> >
> >
> > --
> > Regards,
> > Shalin Shekhar Mangar.
> >
> >
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
>


-- 
Regards,
Shalin Shekhar Mangar.


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message