lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Luiz Armesto <luiz.arme...@gmail.com>
Subject Re: Adding and deleting documents in the same update request
Date Sun, 27 Jan 2019 12:50:22 GMT
You're correct. It' not a good idea mixing different operation types in the
same request. You can't rely on the operations order. There is a
presentation about SolrJ where they explain it:

https://youtu.be/ACPUR_GL5zM?t=1985



On Sun, Jan 27, 2019, 09:14 Andreas Nilsson <anilsson@atex.com wrote:

> Thanks for the suggestions, Shawn.
>
>
> Unfortunately in this case, I don't think there is a natural key to use as
> the primary key due to the requirements of having multiple versions of the
> source indexed at the same time.
>
>
> I have now found a way to tweak the delete query in order for it to not
> overlap the added documents. I will go with either that or sending the
> deletes as separate requests.
>
>
> Just as a clarification, however: am I correct to assume that the
> multi-update operations are executed in an undefined order and can fail
> partially when sent like this? It's my leading theory for a bug I am
> investigating at the moment, and seems very likely given what I've seen,
> but it's also very hard to reproduce.
>
>
> Regards,
>
> Andreas Nilsson
>
>
> ________________________________
> From: Shawn Heisey <apache@elyograg.org>
> Sent: Wednesday, January 23, 2019 3:33 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Adding and deleting documents in the same update request
>
> On 1/23/2019 5:58 AM, Andreas Nilsson wrote:
> > Related: in the case where I cannot rely on the operations order in a
> single update request, is there a recommended way to do these kinds of
> updates "atomically" in a single request? Ideally, I obviously don't want
> the collection to be left in a state where the deletion has happened but
> not the additions or the other way around.
>
> Assuming that you have a uniqueKey field and that you are replacing an
> existing document, do not issue a delete for that document at all.  When
> you index a document with the same value in the uniqueKey field as an
> existing document, Solr will handle the delete of the existing document
> for you.
>
> When a uniqueKey is present, you should only issue delete commands for
> documents that will be permanently deleted.
>
> Alternatively, send deletes in their own request, separate from
> inserts.  If you take this route, wait for acknowledgement from the
> delete before sending the insert.
>
> Thanks,
> Shawn
>
>
On Jan 27, 2019 09:14, "Andreas Nilsson" <anilsson@atex.com> wrote:

Thanks for the suggestions, Shawn.


Unfortunately in this case, I don't think there is a natural key to use as
the primary key due to the requirements of having multiple versions of the
source indexed at the same time.


I have now found a way to tweak the delete query in order for it to not
overlap the added documents. I will go with either that or sending the
deletes as separate requests.


Just as a clarification, however: am I correct to assume that the
multi-update operations are executed in an undefined order and can fail
partially when sent like this? It's my leading theory for a bug I am
investigating at the moment, and seems very likely given what I've seen,
but it's also very hard to reproduce.


Regards,

Andreas Nilsson


________________________________
From: Shawn Heisey <apache@elyograg.org>
Sent: Wednesday, January 23, 2019 3:33 PM
To: solr-user@lucene.apache.org
Subject: Re: Adding and deleting documents in the same update request


On 1/23/2019 5:58 AM, Andreas Nilsson wrote:
> Related: in the case where I cannot rely on the operations order in a
single update request, is there a recommended way to do these kinds of
updates "atomically" in a single request? Ideally, I obviously don't want
the collection to be left in a state where the deletion has happened but
not the additions or the other way around.

Assuming that you have a uniqueKey field and that you are replacing an
existing document, do not issue a delete for that document at all.  When
you index a document with the same value in the uniqueKey field as an
existing document, Solr will handle the delete of the existing document
for you.

When a uniqueKey is present, you should only issue delete commands for
documents that will be permanently deleted.

Alternatively, send deletes in their own request, separate from
inserts.  If you take this route, wait for acknowledgement from the
delete before sending the insert.

Thanks,
Shawn

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message