Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Subject: Re: Separating Search and Indexing in SolrCloud
To: solr-user@lucene.apache.org
References: <2c01bba3-8e87-5e25-2150-6724a29f0636@jarekrozanski.com>
 <CANsFX06UUCpFz6gKTxoR44w4BrPwrqseo4XnD1e1SMpHjSnozA@mail.gmail.com>
 <8182d82a-73e4-70b1-96ed-1f3d62e2b7e2@jarekrozanski.com>
 <97a29ebe-4966-63e2-a777-4b23f38791d4@elyograg.org>
From: Jaroslaw Rozanski <me@jarekrozanski.com>
Message-ID: <f644fbb6-b9d4-7654-ad6e-f3b1762f2e0e@jarekrozanski.com>
Date: Fri, 16 Dec 2016 21:05:27 +0000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
 Thunderbird/45.5.1
MIME-Version: 1.0
In-Reply-To: <97a29ebe-4966-63e2-a777-4b23f38791d4@elyograg.org>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="qWave1hRUkvAsAPIMWV4CiD8tNk2NuGV3"
archived-at: Fri, 16 Dec 2016 21:06:00 -0000

--qWave1hRUkvAsAPIMWV4CiD8tNk2NuGV3
Content-Type: multipart/mixed; boundary="m0WMDnkimfl7ec6FSCvO5OFGI16q5n55S";
 protected-headers="v1"
From: Jaroslaw Rozanski <me@jarekrozanski.com>
To: solr-user@lucene.apache.org
Message-ID: <f644fbb6-b9d4-7654-ad6e-f3b1762f2e0e@jarekrozanski.com>
Subject: Re: Separating Search and Indexing in SolrCloud
References: <2c01bba3-8e87-5e25-2150-6724a29f0636@jarekrozanski.com>
 <CANsFX06UUCpFz6gKTxoR44w4BrPwrqseo4XnD1e1SMpHjSnozA@mail.gmail.com>
 <8182d82a-73e4-70b1-96ed-1f3d62e2b7e2@jarekrozanski.com>
 <97a29ebe-4966-63e2-a777-4b23f38791d4@elyograg.org>
In-Reply-To: <97a29ebe-4966-63e2-a777-4b23f38791d4@elyograg.org>

--m0WMDnkimfl7ec6FSCvO5OFGI16q5n55S
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Thanks,


On 16/12/16 20:56, Shawn Heisey wrote:
> On 12/16/2016 5:43 AM, Jaroslaw Rozanski wrote:
>> Leader is responsible for distributing update requests to replica. So
>> eventually all replicas have same state as leader. Not a problem. It
>> is more about the performance of such. If I gather correctly normal
>> replication happens by standard update request. Not by, say, segment
>> copy.=20
>=20
> For SolrCloud, yes.  The master/slave replication that existed before
> SolrCloud does work by copying segment files, but SolrCloud does not
> work that way.  The old master/slave replication feature IS used by
> SolrCloud, but ONLY for index recovery -- copying the entire index from=

> the leader to another replica in the event that the replica gets so far=

> behind that it cannot be brought current by regular updates and/or the
> transaction log.  This is also used to make new replicas.
>=20
>> Hence, if my understanding is correct, sending search request to
>> replica only, in index heavy environment, would bring no benefit.=20
>=20
> Correct, it would have no benefit.  There's something else: when you
> send queries to SolrCloud, they do not necessarily stay on the node
> where you sent them.  By default, multiple query requests are load
> balanced across the cloud, so they'll hit the leader anyway, even if yo=
u
> never send them to the leader.

With custom Solr Client the above logic no longer applies to my case. I
can easily control to which replica/core in shard my query is directed
to (along with distrib=3Dfalse).

>> So the question is: is there a mechanism, in SolrCloud (not legacy
>> master/slave set-up) to make one node take a load of indexing which
>> other nodes focus on searching.=20
>=20
> Indexing will always be done by all replicas, including the leader.
>=20
> Something to mention, although it doesn't accomplish what you're after:=
=20
> There is a preferLocalShards parameter that you can send with your quer=
y
> to keep SolrCloud from doing its load balancing *if* the query can be
> satisfied from local indexes.  This parameter should only be used in on=
e
> of the following situations:
>=20
> * Your query rate is very low.
> * You are already load balancing the requests yourself.
>=20
> If the preferlocalShards parameter is used in other situations, it can
> end up concentrating a large number of requests onto some replicas and
> leaving the other replicas idle.
>=20
> https://cwiki.apache.org/confluence/display/solr/Distributed+Requests#D=
istributedRequests-PreferLocalShards


Yeap, already solved. I am more concerned with indexing memory
requirements at volume affecting performance of search requests and/or
cluster stability.

> Thanks,
> Shawn
>=20


--=20
Jaroslaw Rozanski | e: me@jarekrozanski.com
695E 436F A176 4961 7793  5C70 AFDF FB5E 682C 4D3D


--m0WMDnkimfl7ec6FSCvO5OFGI16q5n55S--

--qWave1hRUkvAsAPIMWV4CiD8tNk2NuGV3
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJYVFcbAAoJEK/f+15oLE097UgH/3Y+9WbYM45Ggqz94CKQ791t
juTT8opoIRubti5x6f+M9l/XWTlNLH30TpX8ojjL24EaEq9S67g+6qsOR9YyPG9h
yY09eRZjbqw81pE5DwtJyZmWPIxOG5DAk7EUEL0BG/lxCWJWXfcviQlyubJqitGR
Lib6JH33/EvPbJMrrgOoVjIx39b0VDLH59FuYP/r8l5Gx8oV9OLJuWHf15/LHYHI
CBHYLePFoc6vPXVsmpALnhwHGf7I16RGw6ELmE7wQZx2c/af4xgjCbNTwOGViGG/
GV9TGiYZzF3kal8GnrpAfaKou+4Y5lMVgAw92SZ2/FGet8UHjfP3y9ihAzFOoGY=
=kDv4
-----END PGP SIGNATURE-----

--qWave1hRUkvAsAPIMWV4CiD8tNk2NuGV3--