Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of primoz.skale@policija.si
 designates 217.72.81.13 as permitted sender)
In-Reply-To: 
 <CAOOKt52Ec4m36-Az64tLeKi5xyh2phgeWVF_ew3_DbOhNebaTw@mail.gmail.com>
References: 
 <CAPkhjsDOhkh1VropWhn48onZGf6Uzz57hyMHMWJYHowX_NE3MQ@mail.gmail.com>
	<CAOOKt52iP6yuwk2nYbJQfyvW-imsFfiF4FRRH9opZY9y=hJ=vA@mail.gmail.com>
	<CAPkhjsB_FFpBBcKVP19Scvs=b725_BFzZ+oEiv644R63W5m25A@mail.gmail.com>
	<CAOOKt52p+8u2Kz3cHwZsoni3qr_iYF7ksZyvHCQ1r8CNeg-e+A@mail.gmail.com>
	<OF6BA11E1B.83046B79-ONC1257C06.00224060-C1257C06.0022B522@policija.si>
	<CAOOKt50rrcP1vzFqgm4Jt--c906sD5PrWinzGi7NiE+QAABqwg@mail.gmail.com>
	<OF898FA680.10E07EAF-ONC1257C06.0028A67B-C1257C06.0028E3A4@policija.si>
 <CAOOKt52Ec4m36-Az64tLeKi5xyh2phgeWVF_ew3_DbOhNebaTw@mail.gmail.com>
To: solr-user@lucene.apache.org
MIME-Version: 1.0
Subject: Re: Regarding Solr Cloud issue...
Message-ID: 
 <OF0F3C4D38.323B2C86-ONC1257C06.003036D1-C1257C06.003072EA@policija.si>
From: primoz.skale@policija.si
Date: Wed, 16 Oct 2013 10:49:07 +0200
Content-Type: multipart/alternative;
 boundary="=_alternative 003072E7C1257C06_="

--=_alternative 003072E7C1257C06_=
Content-Type: text/plain; charset="ISO-8859-2"
Content-Transfer-Encoding: quoted-printable

Yap, you are right - I only created extra replicas with cores API. For a=20
new shard I had to use "split shard" command.

My apologies.

Primo=BE


From:   Shalin Shekhar Mangar <shalinmangar@gmail.com>
To:     solr-user@lucene.apache.org
Date:   16.10.2013 10:45
Subject:        Re: Regarding Solr Cloud issue...


If the initial collection was created with a numShards parameter (and=20
hence
compositeId router then there was no way to create a new logical shard.=20
You
can add replicas with the core admin API but only to shards that already
exist. A new logical shard can only be created by splitting an existing=20
one.

The "createshard" API also has the same limitation -- it cannot create a
shard for a collection with compositeId router. It is supposed to be used
for collections with custom sharding (i.e. "implicit" router). In such
collections, there is no concept of a hash range and routing is done
explicitly by the user using the "shards" parameter in the request or by
sending the request to the target core/node directly.

So, in summary, attempting to add a new logical shard to a collection with
compositeId router via CoreAdmin APIs is wrong, unsupported and should be
disallowed. Adding replicas to existing logical shards is okay though.


On Wed, Oct 16, 2013 at 12:56 PM, <primoz.skale@policija.si> wrote:

> If I am not mistaken the only way to create a new shard from a=20
collection
> in 4.4.0 was to use cores API. That worked fine for me until I used
> *other* cores API commands. Those usually produced null ranges.
>
> In 4.5.0 this is fixed with newly added commands "createshard" etc. to=20
the
> collections API, right?
>
> Primoz
>
>
>
> From:   Shalin Shekhar Mangar <shalinmangar@gmail.com>
> To:     solr-user@lucene.apache.org
> Date:   16.10.2013 09:06
> Subject:        Re: Regarding Solr Cloud issue...
>
>
>
> Chris, can you post your complete clusterstate.json? Do all shards have=20
a
> null range? Also, did you issue any core admin CREATE commands apart=20
from
> the create collection api.
>
> Primoz, I was able to reproduce this but by doing an illegal operation.
> Suppose I create a collection with numShards=3D5 and then I issue a core
> admin create command such as:
>
>=20
http://localhost:8983/solr/admin/cores?action=3DCREATE&name=3Dxyz&collectio=
n=3Dmycollection51&shard=3Dshard6

>
>
> Then a "shard6" is added to the collection with a null range. This is a
> bug
> because we should never allow such a core admin create to succeed=20
anyway.
> I'll open an issue.
>
>
>
> On Wed, Oct 16, 2013 at 11:49 AM, <primoz.skale@policija.si> wrote:
>
> > I sometimes also do get null ranges when doing colletions/cores API
> > actions CREATE or/and UNLOAD, etc... In 4.4.0 that was not easily=20
fixed
> > because zkCli had problems with "putfile" command, but in 4.5.0 it=20
works
> > OK. All you have to do is "download" clusterstate.json from ZK ("get
> > /clusterstate.json"), fix ranges to appropriate values and upload the
> file
> > back to ZK with zkCli.
> >
> > But why those null ranges happen at all is beyond me :)
> >
> > Primoz
> >
> >
> >
> > From:   Shalin Shekhar Mangar <shalinmangar@gmail.com>
> > To:     solr-user@lucene.apache.org
> > Date:   16.10.2013 07:37
> > Subject:        Re: Regarding Solr Cloud issue...
> >
> >
> >
> > I'm sorry I am not able to reproduce this issue.
> >
> > I started 5 solr-4.4 instances.
> > I copied example directory into example1, example2, example3 and
> example4
> > cd example; java -Dbootstrap=5Fconfdir=3D./solr/collection1/conf
> > -Dcollection.configName=3Dmyconf -DzkRun -DnumShards=3D1 -jar start.jar
> > cd example1; java -Djetty.port=3D7574 -DzkHost=3Dlocalhost:9983 -jar
> start.jar
> > cd example2; java -Djetty.port=3D7575 -DzkHost=3Dlocalhost:9983 -jar
> start.jar
> > cd example3; java -Djetty.port=3D7576 -DzkHost=3Dlocalhost:9983 -jar
> start.jar
> > cd example4; java -Djetty.port=3D7577 -DzkHost=3Dlocalhost:9983 -jar
> start.jar
> >
> > After that I invoked:
> >
> >
>
>=20
http://localhost:8983/solr/admin/collections?action=3DCREATE&name=3Dmycolle=
ction51&numShards=3D5&replicationFactor=3D1

>
> >
> >
> > I can see all shards having non-null ranges in clusterstate.
> >
> >
> > On Tue, Oct 15, 2013 at 8:47 PM, Chris <christudas@gmail.com> wrote:
> >
> > > Hi Shalin,.
> > >
> > > Thank you for your quick reply. I appreciate all the help.
> > >
> > > I started the solr cloud servers first...with 5 nodes.
> > >
> > > then i issued a command like below to create the shards -
> > >
> > >
> > >
> >
> >
>
>=20
http://localhost:8983/solr/admin/collections?action=3DCREATE&name=3Dmycolle=
ction&numShards=3D5&replicationFactor=3D1

>
> >
> > > <
> > >
> >
> >
>
>=20
http://localhost:8983/solr/admin/collections?action=3DCREATE&name=3Dmycolle=
ction&numShards=3D3&replicationFactor=3D4

>
> >
> > > >
> > >
> > > Please advice.
> > >
> > > Regards,
> > > Chris
> > >
> > >
> > > On Tue, Oct 15, 2013 at 8:07 PM, Shalin Shekhar Mangar <
> > > shalinmangar@gmail.com> wrote:
> > >
> > > > How did you create these shards? Can you tell us how to reproduce
> the
> > > > issue?
> > > >
> > > > Any shard in a collection with compositeId router should never=20
have
> > null
> > > > ranges.
> > > >
> > > >
> > > > On Tue, Oct 15, 2013 at 7:07 PM, Chris <christudas@gmail.com>=20
wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I am using solr 4.4 as cloud. while creating shards i see that=20
the
> > last
> > > > > shard has range of "null". i am not sure if this is a bug.
> > > > >
> > > > > I am stuck with having null value for the range in
> clusterstate.json
> > > > > (attached below)
> > > > >
> > > > > "shard5":{ "range":null, "state":"active",
> > "replicas":{"core=5Fnode1":{
> > > > > "state":"active", "core":"Web=5Fshard5=5Freplica1",
> > > > > "node=5Fname":"domain-name.com:1981=5Fsolr", "base=5Furl":"
> > > > > http://domain-name.com:1981/solr", "leader":"true"}}}},
> > > > > "router":"compositeId"},
> > > > >
> > > > > I tried to use zookeeper cli to change this, but it was not able
> to.
> > I
> > > > > tried to locate this file, but didn't find it anywhere.
> > > > >
> > > > > Can you please let me know how do i change the range from null=20
to
> > > > something
> > > > > meaningful? i have the range that i need, so if i can find the
> file,
> > > > maybe
> > > > > i can change it manually.
> > > > >
> > > > > My next question is - can we have a catch all for ranges, i mean
> if
> > > > things
> > > > > don't match any other range then insert in this shard..is this
> > > possible?
> > > > >
> > > > > Kindly advice.
> > > > > Chris
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Regards,
> > > > Shalin Shekhar Mangar.
> > > >
> > >
> >
> >
> >
> > --
> > Regards,
> > Shalin Shekhar Mangar.
> >
> >
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
>


--=20
Regards,
Shalin Shekhar Mangar.


--=_alternative 003072E7C1257C06_=--