lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dwane Hall <>
Subject Re: Cursor mark page duplicates
Date Mon, 11 Nov 2019 11:10:39 GMT
Thanks Erick/Hossman,

I appreciate your input it's always an interesting read seeing Solr legends like yourselves
work through a problem!  I certainly learn a lot from following your responses in this user

As you recommended I ran the distrib=false query on each shard and the results were the identical
in both instances.  Below is a snapshot from the admin ui showing the details of each shard
which all looks in order to me (other than our large number of deletes in the corpus ...we
have quite a dynamic environment when the index is live)

Last Modified:23 days ago

Num Docs:47247895

Max Doc:68108804

Heap Memory Usage:-1

Deleted Docs:20860909


Segment Count:41

Master (Searching) Version:1571148411550 Gen:25528 Size:42.56 GB

Master (Replicable) Version:1571153302013 Gen:25529

Last Modified:23 days ago

Num Docs:47247895

Max Doc:68223647

Heap Memory Usage:-1

Deleted Docs:20975752


Segment Count:43

Master (Searching) Version:1571148411615 Gen:25527 Size:42.63 GB

Master (Replicable) Version:1571153302076 Gen:25528

I was however able to replicate the issue but under unusual circumstances with some crude
in browser testing.  If I use a cursorMark other than "*" and constantly re-run the query
(just resubmitting the url in a browser with the same cursor and query) the first result on
the page toggles between the expected value, and the last item from the previous page.  So
if rows=50, page 2 toggles between result 51 (expected) and result 50 (the last item from
the previous page).  It doesn't happen all the time but every one in five or so refreshes
I'm able to replicate it consistently (and on every subsequent cursor).

I failed to mention in my original email that we use the HdfsDirectoryFactory to store our
indexes in HDFS.  This configuration uses an off heap block cache to cache HDFS blocks in
memory as it is unable to take advantage of the OS disk cache.  I mention this as we're currently
in the process of switching to local disk and I've been unable to replicate the issue when
using the local storage configuration of the same index.  This maybe completely unrelated,
and additionally the local storage index is freshly loaded so it has not experienced the same
number of deletes or updates that our HDFS indexes have.

I think my best bet is to monitor our new index configuration and if I notice any similar
behaviour I'll make the community aware of my findings.

Once again,

Thanks for your input


From: Chris Hostetter <>
Sent: Friday, 8 November 2019 9:58 AM
To: <>
Subject: Re: Cursor mark page duplicates

: I'm using Solr's cursor mark feature and noticing duplicates when paging
: through results.  The duplicate records happen intermittently and appear
: at the end of one page, and the beginning of the next (but not on all
: pages through the results). So if rows=20 the duplicate records would be
: document 20 on page1, and document 21 on page 2.  The document's id come

Can you try to reproduce and show us the specifics of this including:

1) The sort param you're using
2) An 'fl' list that includes every field in the sort param
3) The returned values of every 'fl' field for the "duplicate" document
you are seeing as it appears in *BOTH* pages of results -- allong with the
cursorMark value in use on both of those pages.

: (YYYY-MM-DD HH:MM.SSSSSS)), score. In this Solr community post
: (
: Shawn Heisey suggests:

...that post was *NOT* about using cursorMark -- it was plain old regular
pagination, where even on a single core/replica you can see a document
X get "pushed" from page#1 to page#2 by updates/additions of some other
doxument Z that causes Z to sort "before" X.

With cursors this kind of "pushing other docs back" or "pushing other docs
forward" doesn't exist because of the cursorMark.  The only way a doc
*should* move is if it's OWN sort values are updated, causing it to
reposition itself.

But, if you have a static index, then it's *possible* that the last time
your document X was updated, there was a "glitch" somewhere in the
distributed update process, and the update didn't succeed in osme
replicas -- so the same document may have different sort values
on diff replicas.

: In the Solr query below for one of the example duplicates in question I
: can see a search by the id returns only a single document. The
: replication factor for the collection is 2 so the id will also appear in
: this shards replica.  Taking into consideration Shawn's advice above, my

If you've already identified a particular document where this has
happened, then you can also verify/disprove my hypothosis by hitting each
of the replicas that hosts this document with a request that looks like...


...and compare the results to see if all field values match


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message