Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of vinay.pothnis@gmail.com
 designates 209.85.214.171 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAN4YXveapJR2etF5gHjOVNNVaTu1hTbsorChXiRimvmveGunWA@mail.gmail.com>
References: 
 <CAKgXx6+Kp7Js=RLuV_mnVWQ17VzdnGm0ZcvtKZ48+u_3nDTn=A@mail.gmail.com>
 <CAN4YXveapJR2etF5gHjOVNNVaTu1hTbsorChXiRimvmveGunWA@mail.gmail.com>
From: Vinay Pothnis <vinay.pothnis@gmail.com>
Date: Tue, 19 Feb 2013 18:12:11 -0800
Message-ID: 
 <CAKgXx6+-q-fvyi+yNK0bKw9X=Da2BmKCzpBngrupXCwjZvS7fw@mail.gmail.com>
Subject: Re: [solr cloud 4.1] Issue with order in a batch of commands
To: solr-user <solr-user@lucene.apache.org>
Content-Type: multipart/alternative; boundary=bcaec54c52529e020a04d61e7d5a

--bcaec54c52529e020a04d61e7d5a
Content-Type: text/plain; charset=ISO-8859-1

Thanks for the reply Eric.

* I am not using SolrJ
* I am using plain http (apache http client) to send a batch of commands.
* As I mentioned below, the json payload I am sending is like this (some of
the fields have been removed for brevity)
* POST http://localhost:8983/solr/sample/update
* POST BODY
  {"add": {"doc":
{"field-1":"1359591340025","field-2":1361301249330,"doc_id":"e.1.78"}
},"add": {"doc":
{"field-1":"1360089709282","field-2":1361301249377,"doc_id":"e.1.78"}
},"delete": { "id": "e.1.78" }}

The evidence is from the logs on the 2 shards.

The following is the log on shard 1:
*INFO: [sample] webapp=/solr path=/update params={} {add=[e.1.80, e.1.80,
e.1.80, e.1.80, e.1.80, e.1.80, e.1.80],delete=[e.1.80]} 0 48*
*
*
The following is the log on shard 2:
*INFO: [sample] webapp=/solr path=/update
params={update.distrib=TOLEADER&wt=javabin&version=2} {delete=[e.1.80
(-1427453640312881152)]} 0 2*
*Feb 19, 2013 6:04:34 PM
org.apache.solr.update.processor.LogUpdateProcessor finish*
*INFO: [sample] webapp=/solr path=/update params={distrib.from=
http://10.10.76.23:8983/solr/ch-madden/&update.distrib=TOLEADER&wt=javabin&version=2}
{add=[e.1*
*.80 (1427453640314978304), e.1.80 (1427453640338046976), e.1.80
(1427453640342241280), e.1.80 (1427453640346435584), e.1.80
(1427453640349581312), e.1.80 (14274*
*53640351678464), e.1.80 (1427453640353775616)]} 0 41*

As you can see, shard 2 gets the "delete" command first and then the
"add/update" commands.
I am sure I have waited until the commit happens. And besides, I am also
using the "softAutoCommit" at 1 second. So, the query results should be
updated quite quickly.

Any pointers would be very helpful.

Thanks!
Vinay


On Tue, Feb 19, 2013 at 5:57 PM, Erick Erickson <erickerickson@gmail.com>wrote:

> Hmmm, this would surprise me unless the add and delete were going to
> separate machines. how are you sending them? SolrJ? and in a single
> server.add(doclist) format or with individual adds?
>
> Individual commands being sent can come 'round out of sequence, that's what
> the whole optimistic locking bit is about.
>
> I guess my other question is what's your evidence that this isn't working?
> Are you just querying your index and looking at the results? If so, are you
> sure you're waiting until after any autocommit intervals?
>
> Best
> Erick
>
>
> On Tue, Feb 19, 2013 at 2:23 PM, Vinay Pothnis <vinay.pothnis@gmail.com
> >wrote:
>
> > Hello,
> >
> > I have the following set up:
> >
> > * solr cloud 4.1.0
> > * 2 shards with embedded zookeeper
> > * plain http to communicate with solr
> >
> > I am testing a scenario where i am batching multiple commands and sending
> > to solr. Since this is the solr cloud setup, I am always sending the
> > updates to one of the nodes in the cloud.
> >
> > e.g.: http://localhost:8983/solr/sample/update
> >
> > *example set of commands:*
> > {"add": {"doc":
> > {"field-1":"1359591340025","field-2":1361301249330,"doc_id":"e.1.78"}
> > },"add": {"doc":
> > {"field-1":"1360089709282","field-2":1361301249377,"doc_id":"e.1.78"}
> > },"delete": { "id": "e.1.78" }}
> >
> > When I include deletes and updates in the batch, sometimes, the order of
> > the commands is not maintained.
> >
> > Specifically, if the document does not belong to the shard that I am
> > communicating with (lets say shard-1), then shard-1 sends the commands to
> > "shard-2". In this case, the "deletes" are sent first and then the
> updates.
> > This changes the order that I originally sent.
> >
> > Any inputs on why the order is not maintained?
> >
> > Thanks!
> > Vinay
> >
>

--bcaec54c52529e020a04d61e7d5a--